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The Rfesearch on l^aiuatiori^roVram is i Northwest Regipnal 
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^and trailing designed to create new evaluation methodologies' for 
use in education.^ This documer^ is one "bf a series oi papers ^nd 
reports produced by program staff , visiting scholars, adjunct ^ 
scholars, and project collaborators-—all^ members of a cooperati 

.network of coll^a^es working on the . development of new 
methodologies. r . 

Hov^.caji mathematical modeling be used to assist in the solution ' of ^ 
complex educational problems?. Whjat ^plications ^ have been made of 
these tecj^riiques in applied setting^? ^These and related issues 
are discussed at lengbh^in this volume on the use of multiple 
alternative s'aAalys is in edu^ational-.decision making. This vo^Lume 
discusses the 'nature and use of math^^jiical models, and how they 
are constructed and implearferited jL^|%)rartic educational settings-* 
An extensive example of the appdication* of these procedures in * 
detenjiining fiscal cutbacks an^ program terminations during the 
reduction of '^ducatiorial funding is also provided here. 
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INTRODUCTIOtj AND PROLOGUE 

*»You iare about to embark upon (ine of the more worthwhile read- 
'^ihg exercises' of your educational and administrative career. 

This book contains an Exposition of a mathematical technique 
>for the evaluation and decision-making of some of the most complex 
issues- facing, professional educators today — at all levels, from 
the building principal- and district superintendent, to the .state 
office superintendent and federal office director. The reader 
IS advised and forewarned, that understanding the information 
found within this text will n^ot proceed without conscious effort 
on the part of the individual wishing to understand the technique 
described within this text. 

. Considered (hopefully) a more readable and diversely applica- 
ble sequel to its predecessor: ' 

THE DESIGN, IMPLEMENTATION AND EVALUATION OF 'MATHEMATICAL 
MODELING PROCEDURES FOR DECISIONING AMONG EDUCATIONAL 
ALTERNATIVES (University Press of America, 1980; 474 pp.) 

which dealt totally with the issue of decisioning school closures, 
this current text will explore in detail the application of simple 
algebraic techniques (conmonly referred to as integer program- 
ming or operations research techniques) to such complex 
educational issues as program reduction, fiscal roll-backs, and 
the matching of computerized hardware/software purchases to ex- 
isting curriculum objectives for computer-assisted and computer- 
managed instruction. Several other applications will also be 
mentioned in a later chapter demoted to current 'drawing-board' 
developmental designs. 
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The intent of. this writing is of course to fecJucate the reader 
as well as provide convincing evidence of the utility oV the model 
illustrated for special evaluation and decisionrmaking within 
what wi'll be called thp multiple alternatives context. In this 
context, potential solutions are viewed as multiple.— that is, 
the solution existing as a series of Several individual actions 
which in combination effect the desire'd resolution of the' identi- 
fied problem. For example, the decision as to which school to 
clos^, and how many, are simultaneous, -interactive-effects il- 
lustrations of the utility of the MAI^ framework. Equally import- 
ant to the reader (as well as this author, obviously), is the 
fact that this book demonstrates the utility of quantitative 
assessment techniques for solving the more complex — often said 

yto be unsolvable — problems faciqg the emerging educational ^ 
✓ . ■ 

professional today. 

Nfultiple Alternatives Analysis for Educational ^valuation and 

Decision-Waking is comprised of four major parts: 
\ 



Part I 

Part II 

Part III 

Part IV 



Introduction to the Modeling Formulation 
Presentation pf a Full Formulation 
Example of a Complete Quantitative Solution 
Future Extensions and Summary. 



wherein each of the underlying eleven (11) chapters are presented 
as stages or phases; of modeling development 'and construction. 

The first , part (Introduction to the Modeling Formulation) 
consjsts of four chapters which discuss the various stages of ' 
modeling development related to: acceptance' of a modeling 
formulation for decision-making, contextual assessment, planning 
fpr modeling development, and lastly, its specific design. The 
specific content for these chapters will address the purpose of 
a modeling framework related to budgeting roll-backs under fiscal 
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crises. Th^ purpose'of, utilizing spepific real-world problems 
to illustrate the application of the model is obvious. ... 

The second part (Pres-entation of a Full Formulation) 1s com- . 
posed of four chapters also; and provides a highly detalfed, 
step-by-step narrative describing tTie multiple- alternatives ^ 
framework as a matching strate-gy for purchasing micro-computer 
hardware and software for CAJ and CMI demands.' These chatpters 
provide information concerning: • the^^itial 'conQe^^A|||feati^ 
of the modeling framework, the development of itst^cm^ru 
characteristics, the format for intended execution, and implemen- 
tation of the model, andHa^tly, the specific approach to be. 
utilized in exploring the resulting solution{s) . 

Chapter Nine is the sole entrant within the tiiird pdrt of 
this manuscript, focusing^ specif ical ly upon thfe^ actual implemen- 
tation of a MAM framework for determining fiscal roll-backs and 
.program terminations during a state of educational funding 
/crisis. The reader is cautioned not, to skip this chapter because 
of its technical nature, or since it is definitively mathematical. 
The presentation is written simply (or simply written, depending 
upon your point of view); and though requiring^the reader to ex- 
tend some effort, it is illustrative of the 'beauty'"' of the model 
for complex decision-making which effects the entire school 
district as a total organization. ' 

. Finally, the fourth part of this work consists of two chapters 
which focus upon current extensions under research and development, 
and some parting thoughts concerning the whole process, respec- 
tively. For the individual who intends to actually experiment 
with the approach narrated 1n this text, ^Chapter Eleven .is must 
reading — as of course (you realize) are the first. ten chapters. 

Good Reading! ! 

Brent E. Wholeben 
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INTRODUCTION TO THE MODELING FORMULATION 
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ft CHAPTER 01 . 

INTRODUCTION TO THE MODELING FORMULATION- 



. I Modeling as a Strategy for Evaluating 
Criterion-Referenced Multiple Alternatives 



■ ■ ' INTRODUCTION TO CRITERION-REFERENCED . ' . ■ 

• - MULTIPLE ALTERNATIVES MODELING 
•> — T. ' 

.Educational decision-making has evolved Into a most complex 
' " . . 1 

and demanding process. What was, once a realm almost complet'e^ly 

associated with 'experience and *arm-chair reckoning*,, administra-' 
tive^perogatiye now demands a highly informed/ structuraTand 
often equally complex approach to problem remediation. E^pcaiors 
and educational administrators'. in particular, have over the past 
several years -chosen to ignore the need to develop more sophisti- 
cated decision-making strategies. Now however, the direction is 
clear. ^ Problems representative of desegregation, declining 
enrollment, school closures, distinct consolidation, attendances^ 
boundary redistricting, P'.L. compliance and (education*?; 

perreni al 'nemesis) reduced funding al location ~ taunt 'every 
educational system, from small school districts through state arid 
federal offices. ' ^ . 

Many of today's complex educational issues can be translated 
into what has become known as the ''multiple-alternatives problem" 
(WpoTeben^, 1980a). For example, in evaluating^'several elementary 
school sites for closure, the question is not, "whether si,te-A 
versus siterrB is ctlosed" but rather hbw many sites and which one? 
should be deactivated in order to fulfill: (1) the objectives of 
the required decision (what we will come to call 'constraints') 
and, (2) the needs of the district involved (which we will soon 
learn is the 'conditional vector * ) . Likewise in developing 
sophisticated curricular systems, the specialist discovers that 
many alternative instructional activities exist which could be 
imp*lemented in fulfillment of the requirements for a priori- 
stated instructional objectives (themselves related to desired 
.concept-learning). Obvious resource factors such as time, cost 
and the varyin^g expertise of available personnel militate against 



th^therwise optimal solution of "doing everythin:g" . However, 
the a|:tual problem is much more" subtle. For ex^amp.le, how related 
are activities 'A''and 'BJ iri regard to satisfying some aspect of 
objective A? Is A more cos'fly yet more eff ecti ve, , whi le B is 
'•less costly' but not as effective? Are. both A and B similarly 
.Efficient in terms of time required for presentation and/or 
conclusion of the activity(.s") itself? And even more subtle, does 
the selection of A in terms of some stated objective affect the 
activities that may . be chosen for another objective,, related to a 
different though required concept? . " 

*> «■ ' 

Thus, the multiple alternatiives approach to modeling various" 
complex situations in the educational sector is itself ^a complex 
milieu; wi.th the purpose of designing a thorough, highly- 
struptural decisioning model to adequately assist the educational 
decision-maker in understanding, analyzing and "decision iilig (sic) 
the multiple alternatives' problems faced today. 

i 

The treatise contained' within this present volume concerns 
the expositon and multiple-alternatives interpretation of those 
complex and, highly volatile prebleuis in education, witnessed as: 

'» Given a situation of reduced resource allocatiqn (and 
therefore required reduced expenditure) • across educational 
' units*, how many units will be continued and which ones; 
Subject to the budget -being balanced and the goals of the^ 
school (district) max^injized. . . . whi >e (of course) minimizing 
any perceivable negative impact upon the system as a whole. 

*urrU =: alternative program budgets for funding; . • 
alternative school- building sites for closure; 
alternative curricular activites for implementation^ 
alternative matches for computer hardware and soft- 
^ ware fd^ CAI/CMI instruction; and 

alternative assignment patterns for attendance areas. 



A final word (of caution) is, required at this point- for the 
reader.. ^ ^- . ; 

'--Educational decision-making today 'is. a tricky business,^ full 
of hidden^ agenda and unforeseen pitfalls. The responsible 
deci5;}on-rnaker views a complex issue as ftherefore) complex; and* 
does not subscribe to the overused adage, "simple solutions to 
complex problems^ should be your objective." Obviously, the 
problem solver cannot attach issues, by "mak^ing mountains- out of' 
molehills", but^frtust nonetheless recognize each "molehill" as a 
^particuTate-sburce of a "new mountain.", 

^ It is not. the hidden goal of the author to convince the 
reader, that^the multiple alternatives approach to certain educa- 
tional problems is the perfect solution.' hlowever, the "MAM" 
is strongly ^Nggested to be one of a minority of preferred tech- 
niques which the emerging educational administrator must be aware 
of and rudimentari ly understand. 
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VISIBILITY. RESPONSIBILITY AND ACCOUNTABILITY in DECI3IONING 

Education is, bombarded daily with' demands for evaluative 
decision-making and dlear-cut answers to those dilemmas current Ty 
facing the'schools. Declining enrollments, schools closures, 
decreasing fiscal reserves and fund,ing bases, personnel lay-offs, 
inability of high students to read, '^4te and compute at desired 
proficiency standards, desegregation and bussing, inner-city 
flight arid suburban blight, grade-level reconfigurations for re- 
distributing certain age-segments of children fW maximized pro- ^ 
gram coverage, and fiscal program rol 1-backs and instructional ' 
program discontinuations — all focus upon a very simple point v 
that most educational administrators have failed to recognize 
throughout the past two decades: 

valid and reliable- sblutions to com'plex issues" 
occur 1-f and only If the evaluation and decision- 
making procedures to develop those solutions wei^'e 
themselves valid and reliable. 

Although, the; reader might • acknowledge the above^ rumination as 
a rather trite corrment, it nevertheless can no longer be ignored, 
that education is i Il-prepared- to deal, with many of the pV-oblems 
facing it today. Current problems associated with school closures 
or fiscal rol 1-backs^^e extremely complex (admittedly.' also trite), 
•yet administrators, sti'll cling to that age-old adage: 

"Don't make mountains out of mole hilVs!", 

sharing in the belief that complex problems do not require complex 
evaluations and decision-making strategies. This is not a belief 
shared by this author, however. 
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The Age of ^gopplexo 



With the advent of computerization ^^--^ and their wide-spread 
acceptance and utilization during the most* recent decade — the 
notion of complSMjty is no longer a barrier to inforkied decision- 
making within the educational sector. Even in terms of the simple 
and very.,affordable micro-computer (Including APPLE and TRS-80),> 
sophisticated evalu^^tion and decision-oriented software packS^ges 
are avail able and nTbreover highly usable, without a great deAl of 
prior training or instruction (e.g. DB MASTER ~ a management " 
information system database gerferator; or vrSICALC,-- an arithme- 
tic worksheet computational scheme for quantitatively-oriented 
nalysis needs). ' 



Whereas the issue: of complexity was often a legitimate reason 
for not being abTe to solve explicated problems as recent as 
the early 1960*s, it has today become a oft-used hedging-technitjue^ 
behind which unsophisticated and uninformed 'aidminstrators attempt 
to hide from their responsibilities as educational leaders. 

■ )• 

One only has to survey the various colleges and universities 
where educational, administrators are trained and certified, to 
obtain an accurate^'picture of the reasoh for decay and retrench- 
ment within education leadership today. Ability . to understand 
and apply statistical analysis, research methods and evaluation , 
procedures are viewed as academic course requirements — rather 
than as knowledge and/or practitioner prerecjuisites. Many admin- 
istrator training programs do 'not even require formal courses in 
evaluation techniques for complex issue decision-making -- or for 
that matter, even simple issue decision-making. Seldom are there 
requirements for formal courses pertaining to. the understanding 
of computer applications to education. 

None of these charges should be of any surprise to the reader, 
since many teacher training institutions have yet to offer formal 
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coursework pertinent -to. the role of micro-computers in the class- " 
room — for both computer-assisted and/or computler-managed in- ^ 
structiUn ~ while the schools are purchasing jnitrocomputer.s °^nd 
packaged 'CAI/CMI software at a. fantastic rate. The age of 'com- 
plexity has truly changed from a time wherein cor^plex question^—- 
and. answers were analyzed via comp.lex procedures, to a time (now) 
where the"^dea- of comolexity deals more with understanding^ "why 
we do whatever it is we do". 

> 

ft ^ ^ ^ 

■ I 

The Need For Visibility ' 

With today's knowledge of new ways to attack and resolve old 
prolflems, and the computerized hardware and software to achieve 
implementation of these new ways, emerging educational adminis- 
trators can (and must) become 'leaders in their field. They must 
take the initiative necessary to evaluate complex issues with' an 
open mind — free of the politically -oriented nonsense which has 
brought the educational professional to the lowest rung on the 
Pfcpfessional ladder a^ far as prestige and trust,.'aF.e con^k^rned". 
Today's administrator must come to understand not only the problems 
within, education today — but also their beginnings as well as 
the.iii(*. intended or required solutions To not do so, is certainly 
the choice of malfeasance and ignorance over progress and direction 

■Visibility is the key. Today's educational leader must not 
only be visible to the school and community served,' but must in 
fact allow problems within that context to surface and be examined 
openly and totally for potential resolution. Attendance at board 
meetings, community organizational functions, and neighborhood 
meetings does Vittle to promote visibility of any significance. 
That visibility which is of concern here, is the creation of an 
environment within which valid and reliable evaluation can be 
performed, and withjn which informed decision-making can thus be 
possible. 

r 
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The Need For Responsibility ' . 

Forgoing a milieu constructed upon the principals of openness 

and visibility for profcjftem-exami nation and problem-solving, will 
produce little more^ than chaos and randomized schizophrenia 

) unless there is strict qr^er and direction to the process.' The 
address of critical problem issues requires what many philosophers 
(for lack of another term) have called the * team-player* approach. 

1 Unfortunately, team-players in the strictest . sense follow the 
rules of their team— and in most games of contest, there are at 
least two sides. One only has to review the proceedings of many 
'management-union negotiation meetings in order to derive^ a 
perspective of how two groups striving for* the same ends can in 
fact destroy the process — and often the ends sought. Unfortun- 
ately, many of today's educational ci^ises can not (and moreover 
will not) be patient for such groups to resolve their political 
and mutual differences. 

Evaluation and decisioning teams are no different. Members 
form sociometrical sub-teams though dnly a single, . formal team 
exists. The need to initially identify direction, focus and the 
medium for project control can often consume several hours of 
meeting 'time because of delays associated with personal differ- 
ences. The end-result of an evaluation project is not the exist- 
ence of team harmony — it is the solution of the identified 
problem. Team-harmony is but the process. 

The key is responsibility. Educational administrators as the 
key decision-maker_of the organizational must be responsible not 
only to the members of the evaluation team, but also to the 
constituents of the particular educational organization involved. 
This educational leader must not only be responsible to the 
conmunity, but must' also portray responsibility regarding the. use 
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and- acceptance qf the evaluation-. team, ftseHf. J-Jhfle the evalua- 
•> tion process must be open and^ visible, so also must the workings-; 
of that team be left unencumbered by the adminstrator. Though 
certain decision alternatives Wy be uncomfortable or personally 
undesireable, tjne team must be\left — moreover, encouraged— tb 
expltjre each and every potential problem resolution (^d'its ^ 
associated procedure). J . ' 

\. ^ . I ■ _ 

The Need For Accountabilitv. . „ 

Finally at stake in the evaluation and problem resolution 
process," is the issue of accountability. Whereas visibi 1 ity gre- 
par^ the milieu for pcdblem resolution, and responsibi 1 ity is 
the primary ingredient for performing the problem-solving activ- 
ities, it remains the role of accountability to prepare for final 
acceptance' of the decision(s) made. 

However, accountability can not exist without' visibility and 
responsibility as necessary allies. The visibility of an open 
process, available for all to scrutinize; the responsibility 
of executing a valid and reliable process in which various solution 
alternatives are explored, examined and evaluated; and preparation 
of materials and documentation demonstrating specifically why one 
solution wa^ selected over all opponents — signals the successful 
environment ;f or valid and reliable decision-making. 

The Applicability of the Multiple Alternatives Model 

As the reader progresses through thj.s text, it will become 
more and more apparent as to the utility of the MAM framework in 
addressing, each of the above issues concerned with visibility, 
responsibility and accountability. Complex issues demand complex 



-complex evaluation techniques, and oftef equal ly, complex 
solutions. iWlthout a sophisticated technique to "address these 
problems, -solution identification and development can be extremely 
tedious, and open to errors of iriany kinds. At the. same time, if ' 
the sophisticated solution generation technique is neither 
understandable nor visible to the gi^up authorizing the stl^dy, 
the^olution resulting will ne'v^ be accepted (embraced?) by the 
individuals r^lquiring the identifi^ed problem's solution. 

Becaus.e of MAM requirements for strict criterion definition 
and measurement- scaling; and beca^jse'each and every poterrtial 
alternative solution is available for checb and rechec1<. as the " 
system progresses toward final splutlbn — the multiple alterna- 
tives modeling technique is highly desireable as a strategy for 
sdlving^ complex problems of the multiple alternatives* type.* 



SIMUI^ATION TO STUDY FUTURE DECISIONING ' IMPACT 



• Before we move into the remaining three topics of this first 
chapter, '.and develop the general, underlying thesis of the MAM 
approach some time should b6 ^spent indoctrinating the reader ^ 
to the role of simulation within evaluation an^NJ^c\ion-lTiaking. 

Simply. stated, simulation means to "try something before ac- 
tual ly, doing it" . The' act of; trying-out an idea is certainly not 

- a. revolutionary approach to problerfi-sol.ving. However, some 
problems are of such a critical nature, that implementing a par- 
ticular solution and exam-ining its effects may result in other 

. prbbl.enis 'completely unforeseen to the original problem-sol verfs. 

" Simulation i.s a means by which the use of criterion variables 

/which are accepted as indicative of certain environment charac- 
teristics, can assfume values which are predictable of the extent 
to which certain types of solutions will be successful. Besides, 
tlie use of criterion indicators as 'impact predictors', criterion 
can also .be .ut 11 ozed to compare various potential solutions in 

^prder to..? elect, a certain subset of these possible strategies as 
'the individual, solution(s) to execute in the face of the identi- 

. fled difficulty.; By merging the use of criterion variables to 
'evaVuat^^M^^ identified potential solutions, with the use 

. of 'crit^riQn indicators for predicting environmental impact due 
to the varipUs /combinations (and permutations) of * the potential 
strategies* being examined -- the ability to simulate the future 
of the org.anization, based upon certain resolutional strategies, 
IS. possible. • 

Not only does the multiple alternatives modeling framework 
prpmbte "such simulative techniques, but it also all ov^. the evalu- 
^^tor/decision-maker to vary the model's requirements for finding 



a solution (called the constraints for^the process) and thus 
provides a format for examining a wide-range of possible solution 
sets' based upon certain, measureable pre-requisites. 

As will be illustrated within this text,- the ability to input 
measureable characteristics of not only the potential solutions, 
but also tfie criterion indicators of the environment make the 
MAM system one of the most powerful evaluation and declsioning 
tools available to the educational leader today. 
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CRITERION REFERENCING AND CONTROL 



Decisions may be rather pragmatically viewed as the results 
of systematic and reality-based evaluation processes. To attain 
the desired ' ictealogical' ends suggested by such goals as visi- 
bility, responsibility and accountabi lity, .the decision-maker 
must rely heavily upon various indicators which will attempt to 
assess the situation which requires a decision be made, measure 
the alternative methods and means with which to formulate a valid 
decision to meet that situation, and evaluate the expected impact 
upon that same situation of the remediation strategy selected. 
Thus, some means for determining the needs and demands associatecj. 
with some problem, the comparative value and strength of various 
alternatives solutions or -remediation strategies to address the 
problem issue(s)," and the simultaneous effects of the solution 
to be executed must be defined. 

Criteria as _a Reference for Determining Value 

A criterion is a standard or^rule by which a potential 
decision can be tested for its value in addressing the proposed 
issue; and a rational judgement' result. A criterion may be as 
objective or subjective as the evalua^or will allow; and either 
quantitative or qualitative in terms of its 'visual' measure. 
Since no decision or judgement is possible without some selected 
determination of value based upon "need, demand, appropriateness, 
Walue, worth, expectation, etc., it is ridiculous to assume, that 
decisions can be, made without reliance upon 'hard performance* 
indicators of worth, nor that complex issues can be addressed 
without the conscious measure, valuation and determination of 
strength, of the complex solutions which may be required. 
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The use of criteria for determining worth and strength is 
not relegated solely to the area of meas'uring, and thereby con- 
trolling, the content of indicators which will be used within a 
necessary decisioning situation. Equally Important moreover, is 
the reliance upon various criterion measures for controlling the 
process of the decisioning procedure itself. For example, the 
evaluator may define several criterion indice of effectiveness 
and efficiency in order to compare various alternative methods 
for resolving the problem situation at hand; and subsequently ■ 
measure these performance indicators in such a manner as to with- 
out doubt produce valid measures via rel iable techniques. The 
process-control issue manifests itself in the situation whereupon 
having measured each criterion across all alternative actions, 
the problem becomes to determine v/hich alternati ve(s) are most 
1 ikely to produfce the desired impacts upon the system without in- 
troducing additional 'new' problems to related system components. 
No single or multiple decision(s) will provide total positive 
benefit to the system, without likewise introducing some amount 
of negative by-product, — there is both good and bad in all 
decisions, and the extent of these factors must be observed and 
controlled for in the decisioning situation. 

■ ■* 

Criteria as _a Reference for Control ling Evaluation 

The issues involved in the control of evaluation procedures 
address two specific areas: the control of positive and negative 
influence expected via the adoption of various alternative solu- 
tion strategies; and the control necessary for determining the 
necessary 'amount' of solution required to effectively solve the 
problem being analyzed. 

The control of positive and negative impact to the system as 
a whole, based upon the positive and negative influences associated 
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with a particular solution (or- set of solutions), is directly re- 
lated to the i^a of preference and trade-off in decision-making. 
We "prefer" to have our decision affect the §ystem in specific 
-ways, and we ."prefer" to have the soloition exhibit certain other 
qualities which may directly or indirectly related to the 
problem being resolved. However, -we are also realistic enough to 
understand that. 'some 'give and take' ^111 be necessary for the 
final acceptance of a particular solution (or set of solutions). 
Such "trading-off " manifests itself in many ways, but is normally 
recognized in two particular instances. In a differentiated 
sense, the decision-making will define certain limits for each of 
the criterion variables which will be utilized in determining the 
value of a particular solution. Such individual limits will 
normally be concerned with only expected negative impact to the 
system based upon that particular alternative solutions being 
selected as the final decision. If such limits are placed upon 
positive Impact, they will always be: defined as 'lower bounds' 
or 'basal minimums' which the particular alternative must surpass. 

The second instance regarding a "trade-off" manifestation^^ 
addresses the integrated sense in controlling for simul^a^^s ^ 
positive and negative impact to the system, based upon»the final 
solutions selected. For this requirement, collective rather than 
individual impact ttD the system based upon the defined criterion 
Indicators will be assigned limits. Therefore, a final group 
of decisions may be viewed as acceptable, based upon not only 
their high positive -impact to the system, but also their low 
negative effect. In fact, some solutions may be more acceptable 
based upon a low negative impact and a medium positive effect, 
as compared to a solution which projects high" positive .benefit 
while presenting some unacceptably medium negative impact to the 
same system, or oneof its inter-related component^. 

The necessity of some integrated-f orm of evaluation control 
is even more predominate in a situation, where multiple solutions 
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will be required to remediate the final problem. The final set 
of , solution\ (final decision) selected will exhibited^an often 
widely diverse range of positive and s^gative impact to the 
"system. Diff erenti al ly, . each final solution will satisfy the 
minimum positive requirements and the maximum negative limits 
set a prior by the evaluator. Integrally, the combined positive 
and negative effects"' associ ated with the full range of solutions 
will in total be acceptable to the decision-maker, and therefore 
to the system as a whole. 

Defining Multiple Criteria in the Generic Sense 

The necessity to define and measure various criterion- 
•referenced indicators for controlling the evaluation and deci- 
sionmaking framework (s) is witiiout question (?) one of the most 
rigorous and demanding of all evaluation skills. It has been 
approached many ways by as many people — sometimes systematically 
and sometimes not — ^but always with specific requirements in 
mind. It remains these 'i?equirements * that ha^^e of ten^esul ted 
in the misuse or abuse of the criterion-referenced process for 
decision-making; and presents the most formidable obstacle for the 
acceptance of a^criterion-ref erenced process in evaluation. 

While few individual's would state, that criteria are not 
necessary for responsible, etc. Svalwation and subsequent 
decision-making, many people ivi"! 1 take the position, that cri- 
terion determination, definition and measurement are impossible 
tasks, fraught with the "uncontrollable dangers" of invalidity 
and unreliability. It has continued to be of curious (and some- 
times humorous) interest to the author, that the critics of 
criterion-referenced decisioning denounce those components of 
• evaluation and decision-making which the cri*ter ion-referenced 
model seeks to control and validate. The real obstacle pf 



22 

32 ^ 



course, lies in tbe trust which must exist between the 
'evaluation modeler*, the *decisioner* , and all those 
Vsystemettes* impacted via the decis^pn(s). Basicall}^, only high 
visibility and accountability will forge the trust needed for the 
-final acceptance of the decisions made. Visibility >vill result 
necessarily from keeping both associated and interested parties 
informed of the techniques, procedures and their outcomes. 
Accountability can be much' more precipitous matter however, and ; 
will demand much more than openess, in order to gain the 
necessary acceptance of the evaluation pVocess. 

* • 
Accountability in this sense requires the evaluation and 
resulting decisions to "take into atcount'* all those ingredients 
which are seen to impact upon both the decisioning process, and ' 
upon the effect to the system associated with that decision. 

. Evaluators have forever been confronted with the question (or 
ploy): "But what if this criterion was taken into account ... 
would there be a .difference in the particular solution selected?". 
Therefore, decision-makers must be prepared to: first , incorporate 
all criterion references which are expected to impact upon the 

.solution to be evaluated; and second , introduce further measures 
on the request of other individuals, in order to examine whether ■ 
any differential impact upon the solution^ chosen results. 

In order to begin the criterion definition or generation ^ . 
process, the evaluator must commence wi/th a somewhat general idea 
as to what references will be necessary to adequately address the 
issue .of utilizing performance indicatprs to measure the full set 
of prospective or potential alternative remediation strategies. 
These general references, or what can/ be referred to as the 
generic set of criterion references, /wi 1 1 provide a theoretical, 
modeling base by which the evaluator/ will be able to define more 
specific (iLe. species-related) criterion indicators for direct- 
measurement. For most purposes, this generic set will .be comprised 
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able to * construct* a measure ofrxperf ormance, simply by 
the current demand- placed upon the system; and the 



of two subsets: generics which model the system for which the 
resulting decisions , are necessary;^ and generics which model the ' 
various alternative solutions which have been formulated to solve 
,the problem(s) associated with that system. 

System-generic criterion-references can be defined as those 
measures of need and demand , that is, the need of the system for. 
some form of remediation based upon certain measures of criteria 
which will demonstrate that need; and the demand of the system for 
relief — their perception of what the system needs ~ for direct 
comparison to the (hopefully) more objective measures of need. 
It should br^ noted, that some evaluation situations will reqCiire 
an addit^nal and separate measure ^,of performance (i.e., current . 
functioning) in'order to directly understand the relationship of 
demand a,nd need in the generic sense. However^, most situations 
will be 
comparing 

perception of need which exists from within the organization qr 
-system as a whole. 

I ■ ' ' ' ■ 

. ^ Sol uT\i on-generic criterion-references can be cfefined as t(1?)se 
'performance' measures of effectiveness , efficiency , expenditure , 
and satisfaction ; Generally, the comparative analysis of 
prospective decisional alternatives will require measures of:. 
(1) what the strategy or tactic will accomplish; (2) how the 
alternative will resolve the dilemma; (3) how many resourco^ must 
be clTnmvtted to implementation; -and (4) how^'satisf led we will be 
with the process and the result. Many species-oriented criteria 
may be required to addre;ss each of th^se four solution-generic 
issues (as well as those for the two (or three) system-generics) . 
The true utility of these, generic indicators lies ^in providing a 
theoretical model for generating the more specific criterion 
measures which-will be input to the evaluation process. ^ 
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An Illustration of the Criterion-Reference Generation Process 

In an earlier work (Wholeben, 1980a), a demonstration of the 
'9 

process for generating necessary criterion forms for evaluating 
schools sites f.or closure, will serve to illustrate the issue of 
defining criterion-references for decision-making. This pafticular 
illustration goes one step beyond the discussion earlier in this 
section, in that the criteria defined for school-site-closure 
decisioning repr^esented a "cross-comparison" between certain 
references utilized in business and industry for site comparisons, 
and their relative usefulness within the educational sector. 

■% 

A total of 34 criterion-references were discerned from a review 
of t?ie relevant business and industrial applications, an(J collapsed 
into six collective categories a§ follows: 

CI3 characteristics of the physical plant 
(9 criteria); . • 

[23 location of the site 
(4 criteria); ,^ 

[3] characteristics of the surrounding neighborhood 
(6 criteria); 

[43 employee (or faculty) information . 
. (5 criteria); 

[53 customer (or student) information 
(4 criteria); "and 

[63 production (or. curriculum) requirements 
(6 criteria). 
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Each of the subsets of criterion references were, defined to matcb 
the cross-comparative relationship between the business or industrial 
section, and the educational domain. The full set, of criterion 
references has been reviewed beginning on the following page. 

The reader will see many indicators of our previously defined generic 
forms: need, demand, performance, effectiveness, efficiency, expendi- 
ture and satisfaction — within the references illustrated. What the 
reader will not witness, however, is a directly discernible process 
, for generating the criterion forms. In other words, no procedure will 
ever replace the knowledge and experience of the evaluator in gener- 
ating indicators for future measurement. A theoretical model based 
upon generic standards will serve only to guide thinking, and hope- ' 
fully alert the evaluator to areas which wi 1 1 require 'representation 
within the evaluation process. No computer program to dateyhas^been 
written to generate criteria for future decisions based upon a few 
entered keywords but we're working on it. 



|( CRITERION CONSTRUCTS FOR TARGETING SCHOOL CLOSURES j| 



(Relating 'Industrial and Educational Domains) 



CONSTRUCT #1: THE PHYSICAL PLANT 



[SIZE OF FJRM OR BRANCH] • \ 

<Industry>, size of firm or .branch reltive to current 
consumer demand , ' 

<Education> capacity of educational site and the 

current/potential enrollment trend(s) in 
the attendance area 



[TYPE OR CONSTRUCTION! ,n / 

* • ■■ ■ ' * ■ ■ ' 

<Industry> . relationship of physical construction to 
* other existing .'.criterion yariabTes 

<Education> characteristics of plant construction in 
. relation to remaining, criterion estimates 



^ V. CAGE/CONDITION OF SITE] . ' 

<Industry> relation vof plant age 'and/ current '^^^^ 

of facility to other criterion estimates 
and on-going operations V ^ 

' ^ ' 27 



<Education> impact of school condition and age upon 
facility costs and academic requirements 



[COST OF SITE AND PREMISES] 

<Industry> property costs associated with purchasing 
^and/or leasing current site/premises; and ; 
• subsequent maintenance of site .for future- 

production ' , / .. / ■ ^ 

.<Education> current principal and interest costs 
' \ associated jvith school pliant 



[TAXESJ . ; ' \ ' , 

• <!ndustry> "Icss^to profit margin due to existing ■ 
'regional tax^structure 

<Education> increase to overaiIl;..b^ 

due to tax requi.rernejiti .^;:;, , . ;, >. 

' ' . - ' '^^^^^^ 

[UTILITY RESOURCE REQUIREMENTS! 

<Industry> . demand upon .riaturaT and -cqriye^ted^ 
f or ^productiqn'-ahd ,bperatiph 

' •;<Education> requirements of heating; and lighting 
. utilities . f on; fae,ilcity/qper . 



[UTILITY RESOURCE COST EXPENDITUftESj- . 



•<Industry;> ' current fiscal ■expenditures for. resource 
resource requirements 

<Education> current fiscal expenditures^ for utility 
requirements 



' - EROOM FOR EXPANSION] V;. ■• ^' '' ' 

V . ■" .. ■ .'■ ^ .■ ■ ■■ , ' .j^ 

vs^J^ndustry> feasibility of .site expansion due to ■ ' 
increased product .demaind 

4^""^ " <EdiJca1;ion>;r : amount of ' non-f acll Ity^founds avail abl.b.:" 



*v ' . - ^f<)r ^'construction additforts without off- . 



.;: ;:,s:efcting existing playground area(s) 



) ■; 



[SECURITY RISKJ . \: ■ 

<radustry> characteristics of^p.iant ^^id .surrounding'! 

s area promoting secur1ty,'>of facility ''; 



<Education> degree of school comp^^lBX security fostered • 
by physical construction = and neighborhood 
location ' W v \ - \\ 



■ CONSTRUCT #2: THE LOCATION OF THE FACILITY 



■-•■ClNDUSTRIAL CONCENTRATION] 




<Industry> present a^or .p^^ concentration of 
par.allel Gtpe|^*;t^ within the 

. .p |fn;ar.K^ area under ;ConSi:i;derati on 



;<Education> Vratio 'of „ existing eduo^^^ butlets. 

( school s ) to the lAcal i zed' iupply^^ 
' ^. available 5chool-aged\ Chi 1^^^^ 



[TRANSPORTATION OfSTANCEl ' V^^^ ■ 

<Industry> '^transportation distance of goods arid 

services between production site(s) and 
various supply and dU^ 

• I 

■a - ■ / 

, ' .. b ' - 

^<Education> transportation distance of students witli'inv 
. . assigned attendance zone to respectiy^e-^^^.^^^^i^ 

v :• : -V school site ' V. ■ ■•■^'■•• 



£TRANSPORTAI^O^^;^ COSTS] " : 

' ' <Indijsi;try>. Vincurre^^ of transporting goods from 

centers j-6 distribution outlets^ 



^Education>- vexpenses associ ated xithvr^q^ busing . 

(e.g. special educationy des:^^^ ertc^) 
and remote-residence transpbrti^ 
students (residences beyond- establislj^^^-i 
• ' walking-distance regulations) s ^.-'vV 



■[REMOTE OR RURAL LOCAf. I GN^^^ 

' . xl ndustry> eff ect" of dependence/ upon o requi red . ut 1 1 i ty 
facilities' not immediately avail abTe to a 
remote production center 



<Educat1on> effect of skewed student-residence*' 

(location) distributions'upon' centralized 
school site location within perimeter 
\ ' attendance areas v 



CONSTRUCT #3: THE NEIGHBORHOOD 



[RESIDENTIAL HOUSING AVAILABILITYJ 

<Industry> .potential residential construction and 
effect upon consumer characteristics 

.<Education> potential residential construction and 

.impact upon future school-aged popul ations 



lAREA INDUSTRIES] „■ ' . ' : ^ 

<Industry> compatH'b.# local industries as positively 
"^V influencing the growth (or stable margijn) 

of consumer magnitude / 

'" " <Edu^at\pn> availabi lity of local industries promoting'^v^ 
- ^ V 'family residential magnitudes (e.g. fire: 

and police protection, groceries, churches)^ 



[RELATED EMPLOYMENT OPPORTUNITIESj r'^ 



r'^^^IndustryV ,.,avai labil ity of n'eighboring industries 
■^'..(business, industry, schools) to provide 
. additional income. ppssibil ities for dual- 

employment families 



;^Education> availabilityj of employment;DpporiupNt.ies: , 
for related spouse occupational interests' ' 



.lAUXIIIAR;Y. I^ACILITIES] ' ' 

' . : <I.ndu$try> V /.availability of r&Vated seconda^y interests 
i" ■ "'to;,:consunip^ (e.g. dep^rived area 

regarding heW\' employment so 



;<Education> Extent of school facility* utilization for 
. extra-educational affairs (e.g. community 
meetings, recreation center) 



[ENVIRONMENTAL IMPACTj 



<Industr^. impact of plant site and produc-fion 

■■ operations upon surrounding area (e.g. 
quality air control, extent of vehicular 
traf/i..c, property values, tax bases) 

<Education> relation of neighborhood school to sur- 
rounding residential* area (e.g. property 
• ' values and tax base,::'r^^ construc- 

tion, rel ocat1 on ;df supporting Industries) 



EINTER-INDUSTRIAL P^AKnIN^^ INDICE] 



<Industry> extent- to which surrounding non-competitive 
industry planning foci match current 
research and development data. 
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<E'ducation> use of community and regional (city, county, 
' , ■ ; ' .state) planning data^to assist educational 
. decisioning 



CONSTRUCT #4: THE FACULTY/EMPLOYEE POPULATION 



[ENTREPRENEURIAL AND' EXECUTIVE PERSONAL PREFERENCtj 

* 

<Industry> management-based qbjectives in the design, 
. production and supply of separately, 
manufactured goods 

<Education> curricular and instructional demand of 
/ school administration related to 

professional biases 



[MANAGEMENT AND TECHNICAL STAFF] 

<Industry> avail abi^lity of and present average costs ' 
for management .associated with desired 
production levels 
« 

<Education> chacacteristics and salaries of school 
administrators associated with 
instructional foci and parental demands 



[LABOR] 



<Industry> avai lability and present average costs of 
l.abor associated with desired production 
levels / 
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<Education> characteristics and salaries of teacher 
staffing and emphases associated with 
instructional foci and parental demands 



[PRODUCTION COST COMPETITION^ 

. <Industry> relationship of resource requirments to 
production output 

<Education> allocation of budgetary requirements 

related to class size, teacher. load, and 
curricular attainment 



[EDUCATIONAL FACILITIES] 

j <Industry> availability of external training and in- 
service resources for improving production 
and supply 

<Education> availability of higher education facilities 
for improving staff knowledge and instruc- 
tional techniques 



CONSTRUCT #5: THE CONSUMER/STUDENT POPULATION 



[PROXIMITY OF RAW MATERIAL . SUPPLIES] 

<Industry> accessibility fo raw material and supplies 
.to manufacturing location 

34 



<Education> proximity of students within walking 
distance of school site 



[BUSINESS GROWTH! 

<Industry> current business trends in consumer 

population and related demand potential 

<Education> projected student enrol lemnt based upon 
recent attendance trends 



[CONSUMER DEMOGRAPHIC CHARACTERISTICS! 

<Industry> consumer demographic characteristics for 
future trend forecasting 

<£ducation> trends of ffimlly size, etc., for predicting 
future enro;^Tmenlh^^tentia^ 



[AFFIRMATIVE ACTIONj - '"''^ 

<IndUstry> current (and proposed) level of federal 
guidelines* compliance 

; ^ ■ ■ 

<Education> current perspecti re^Tciting to desegration 
and minorities (including handicapped, 
special education) committments 
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CONSTRUCT #6 : THE PRODUCT MARKET (CURRICULUM & INSTRUCTION) 



EMARKET PULL AND MARKETING STRATEGIES! . 

<Industry> consumer demand (market pull) and manufac- 
. turer's planned focus of supply (marketing 

strategy) 

* . ■ < 

<Education> curricular demand emphases (basics - 
curricula, individualized instruction, 
special education)- of parents in attendance 
area, and the current (and potential) 
av^iilabil ity of such academic activities 
within the school 



[LABOR STABILITY] 

.<Industry> turnover history and/or projected avail- 
ability of technical staff in production 
(labor, etc,)' 

' <Education> trend characteristics of instruction staff 
parallel to established instructional 
objectives 



[FACILITY UTILIZATION] 

<Industry> extent of current production utilization 
of exist i-ng plant resources \ • 

<Education> degree of academic and co-curricular 

utilization of existing school facility. 
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resources '(e.g. class space, specialized 
instructional areas) ''^ 



[PRODUCTION INDEX] r ' . 

<Industry> current rate of production output based 
upon existing resourced and/or facilities 

. <Education> level of academic attainment based upon 
current curricula and instructional 
. ^ Strategies 

[AUXILIARY USE OF FACILITFE^^^- ''■'[:':%■ '^-i' ' 

<Industry> amount .of existing specialized plant 
equipment which is essential to the 
missioni of production 



<Education> existence of specialized f 3<;,ilf;J^1es integral 
to the academic process (sfiefide laboratory, 
swimming pool, built-i.ni A.V." 



[ACCESSIBILITY OF PREMISES^ 




<Industry> avai 1 abi 1 i ty of pt640yij^ 
distribution centfei?^^)^%^ 
centers to consumer ^ > ^rY*^ ^ v ' --'v^ ^ 

<Education> flexibility of schobi^tX?^ 

to incorporate mul ti-^'lm^nsfdfj^*^^^^^^ 
' . interests and instructiJdfnaT^'gq^^^ 



OPERATIONS RESEARCH MP THE EVALUATION OF ' 

FEASIBLE ALTERNATIVES ' ' 

■ Chapter two (following) will devote much of its content to the 
exposition of budgeting and funding as "a structural allocation- 
oriented activity. A position will be taken which will specifi- 
cally adhere to the philosophy that fiscal modeling (i.e. the 
simulation of a fiscal decis>Gning system) must be criterion-' 
Veferenced to the actual (real-lxife) system; and that, these cri-"^ 
teria should be designed in such a way as to perform three vital 
functions: . ^ ^ 

(1) to reliably represent the triie system being. modeled 
(simulated) ; 

(2) to validly represent those factors (inputs, outputs, 
processes) which are required (aind desired) to provide . 
the necessary information in order to make decisions); 

■ (3) to totally represent the\impact to the system (-as a 
whole) of the potential alternative decisions being 
s.evaluated. 

' The i^ma1^|ij^;<()f Chapter one will (premeditatively) focus its 
^^ergiesj^qn pr||a||^^ to view thfe fiscal crisis- 

situation,- .art|-it?^g^r^ for fiscil^roll -backs, as an 

inustratt'ife/|^:a^^ alternatives decision-making. 

';Ifi^ this ca^^ev^pl^^f^a^ are defined as either all possible 
programs (soui^<^';®|pE|0rtditure) or significantly distinct pa?ts 
Of prograips w^fw^^ discontinued and therefore deallocated 

from the existing budget; that' is, rolled-back in order to 
balance the-b'udget. To evaluate these many alternative, poten- 
tial sources of cutbacks!, criteria are required which will not 

38 ' ' 
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Jigiire L Representation of a Seqiiencial, Criterion Referenced Model 
. ... 'for Systematically Developing a Hultiplo Alternatives ' 
Soliitj'pn Set'; ^ ' 
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only describe the attributes of each alternative 1n terms of its 
contributions, to system function (or lack of such contributions), 
but will also demonstrate the costs (object category 
expenditures) associated with each of the alternatives. The 
overall goal then is to select those alternative programs 
(decisions) which may be feasibly and rationally rolled-back 
without [Droviding major detrlment^to. the system's required 
functioning, while satrsfyirtg the reduced budgetary limits ^ 
imposed, by . the fiscal crisis.. '§ ■ . ' 



Optimal Deci.sioninq Within a Constrained Feasible Space 

Any decision, viewed as the best possible alternative course, 
of action to operationWt^e, must by definition be optimal ; that 
is, all things considered, this action posits the best interests 
of the organizations or system being modeled. Simulation of 
these "things" and "interests" results from the use of criteria 
to measure the value of each altern'ative and its |Sipact upon the 
system as a whole — ' that is, how the systejn is constrained by 
these. criterion measures across all alternatives. Such measures 
are referred to as criterion constraints . Those alternatives 
decisions (when^ evaluated) will, display degree of optimality 
("best"r-ness) in addressing the solution to. the problem defined; 
but first, each particular alternative must itself be a reaso- 
nably potential solution to the problem; that is, exhibit the 
quality of feasibility ^ 

The context of decisiotiing afternatives is thus a rather 
interesting flgw from a traditional, needs assessment (What is the 
real^robtem? thjat is, not the system of the underlying problem) 
to the determination of a set of solutions to be implemented via . 
a cr iter i on-referenced' Inodel of^'value and worth, versus impact. 
<Figure 1> schematically depicts this to^viously interesting) flow 

. ' 40 
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After.the real problem is determined, dissected and defined^ 
"(the -prD^:^^^^^^^^^^ and regulations (oper*?iting goals) of the 

• sys;^;#&^i4e-the first set of criteria to/^imijact the simulati\)n. 
Stan'tJardS- Provide the necessary data to' ;a$i^i^^^^ of 
all possible alternative sources of acMoai-the decisibn^maker 
must consider and evaluate (i.e. the .se|: p| random alternatives) . 
Next, established priorities; are defirieH.an^^^^^ into/;a;:,set 
of criteria which al^w further sjutinyjof--^ random 
alternatives, and their measured impact upon the decisioning 
system. Often times, an alternative may be "possible" but not 
"plausible" due to certain es'tablished priorities. Alternatives 
which survive this recent criterion-focused^evaluation become 

■ ? ■ 

known as feasible. Finally, the more important (weidhted) :'6ri- 
teria are drawn into the evaluation in order to focusHhe- optimi- 
zation standards for the decisions about to be made (that ?fs, 
selection of al terne^ti ves) . 

The potential existence of an alternative mix-set focuses 
once again upon the idea of singular versus multiple 
%lternative(s)' frameworks. Recall that a singular framework 
involves the^ choice of one and only one (or none; of course) 
alternative course ^qf action based upon the evaluative criteria 
used^ A multiple alternatives' setting, permits the choice of a 
group of several alternatives that when implemented as a group 
(not necessarily simultaneously), produce the desired process and 
attain the required result. • 
t ■ ■■ . 

Fiscal Allocation as a Multiple Alternatives Problem 

Fiscal crises provide the budget manager and program admi- " 
nistrator with a unique experience, "to accomplish more for * ^. 
less". Though tongue-in-che^ the unfortunate reality of . 
today* 5 economy and our best program prognostications for the 

■ •;■ * • . ■ ' .... 



future point to a steadily decreasirig funding, base., 
vf.unding will. noi. however: be followed by the public's reduced rreed 
fm educationat'services, either tn*(!iijantity or quality. 
Organizational philosophies, goals -and needs will: stay relatively 
constant; yet with a new demandl^fort^ 

demonstrated accountability^'.'After 25 years of no-holds-barred 
development and spending/ can •education *ecjually' meet the new 
demands fqr/a^ roll -backs, in light of declining 

enroTimeh^ school ' closures, a sagging nati^na^. economy ahd the ^ 
ever-increasing clemand by teacher for higher salaries?. 

Whether the problem begone of fiscal allocation or 
deallocation, the ;fundi,ng framework for* prtfgran^, budgeting is a ^ 
mul ti pie a1 te^Kiat i ve modeTi nq problem. - donside^^ : 
determine'which- prograiri^^ 

budgejtary ^imitations; and therefoWv.'vihich programs will hot... 
This^is obviously a decisioning^ situation, whereby the goal is to 
fund as many programs as possible within the prescribed budget, 
based upon: (1) each program's merits, (2) the overall, system's 
-aeeds, and (3) the impact of the, alternatives — individually and 
'collectively — upon system functioning as a whole. Each alters , 
native's merits (type aiid extent) will be measfll^ed via the 
various criteria whiah have been. a pri^ori identified as 
demonstrative of the system's definitipa,^.f,, 'merit ' or 'impact'. 
■ Finally,, the cost for every aspect of ^eac^;;^^ is computed; 
and entered as a measure of impact to the system's' budget, in 
deciding to implement the Drograni (expenditure) or not (savings; 

with an opportunities cost) 3 

^- 

Concisejy'^stat the fiscal allocation between multiple, 

■ ■ . ■ • ■ • 

competing programs assumes th^ f-bllowing direction: 

■■^ To choose , (and therefore also f ai T-to-choose ) some finite 
number of ^ograms from among the available alternatives — 



•F.1 . . F 1 sc a:i ,.v^/?l 1 qcati ons as a Mu 1 1 i p 1 e Alterriat i ve P robi enl, 







Multiple Alternatives 

■ ■; . '■ 




Crfteria 


' . "'*■ ' ' i 


[ P7og2 " ■ 


A 


T~'Trogn 


' ■. . ' • ■• 
pQsUiye,b'ii[)act 1. 


+11 


. +12;;. 


+13 . : 


+ln 


•"; ; y . , r ^ * ^- 

' •* :. . T ■ t 


+21 


+22 ■ 


''•23 


: +2ii 


; .... vv - , , ■■ ,3._ ■ 


X+31 


r -( 

■ +32. ■ 


; +33 . 

■/ ' .'•"> 


; . +3n 


, ' ,1. 

, ■ ■ ■ * -'r ■ ■ 


■ ■ 

..■•-It 










'\r -n. 




■ ■"-23&'v^;' 




■ ' ■ 3._ ■ 




:'''■;•.•«•' -32- 


■J/h-33^:''V': 


" -3h': 


Ijiecifk Costs ,1. 


$11 


' .' 1 ' , ' ' '* ■ ■ 


1. « ■ ^ . . ' . ■ J ■ 


-ilh 




' $21 


$22 • 




V $2n 




$31 


, $32 , 


$33., V • 

. ■ ■../' V,' ■ 


* $3^ 



available 



ERIC 



53 



each alternative associated ivith measures of merit; worth,; 
impact and cbst^-r- suph that: . . ■ • 

■ • •V'^ 1/ total (collective) positive im pact to the 'system is 

' ■ , ■ ■ — — ' — ■ ^ii. V 

^ ; maximized (meeting needs,, goals and intereiits); 

* \, ; " • 2. 'fotal (collectivie)^^negati ye impact is minimized 

. (not meeting. .needs, ^c. ) ; an^. i 

■ . r. 3. total (summed .program budgetary demand does not ' * 
exceed the amount of available monies; 

.0 

- -V . . . • ■ 

<Figure 2>' provides a simple outline of l^e above stated goal(s*). 

^ V -^Whi^fl-e ris^^^^^ traditional cost-benefit analytical frar 

.Rework 4^ <Ftgur6 5> once again affirm^s the 

demand' for -an evaluation tool whi^Ch' js "capable o^^ anaiyz'ihg the ; 
TO le ( s ) -of; mu ] t i p 1 e: alter n at i v6s acr os s mu 1 1 i pi e cr 3 1 er i a : , and ■ 
selecting those alternatives which best_ fit the criterion- , 
constrained -system (decisioning matr;ix). Again, we are faced^ 

with the issue of interact! ve-effects modeii rig.. . , / 

... * ' ■ u *..."■ ■ ■• ■ _ . " 

■ ■■■ ■■ ' ' -^.^ .■ ■ . . ■ ..• . • ^ «' 
\Ijiteract.i ve;Effects' Mq^^^^ " ^ - ■ . „ * " 

. There 'exists an fundamental^ the interac- 

tive nature of^^selecting'fr*om aifjong multiple ajternatives; that 
1.5, the total comb i,ne^ effect of one choice* based 'upon the 
^ various values of each alternative's criterion measures: and the 
desire to cHoqse thatvset- of alterriati ves-.which^ demonstrates a . 
c 0 T 1 ec t i V e c omp 0 s i t e ' of acceptable criterion value's. * This, is 
.complica\ed by the fact that difYerent comBinations of alter-" 
nati^ves>re: possible in forrffing tfjfi final solution ^etl Simply 
(?)^ statea, such a deci.sioning '' requirement iter a nightmare. But ' 
xan a technical' St rat formulated to address equally the 
issues of technique asvWl J as th^, fiscal anocatidlt problem - 
•'itself?'.,*'"-" /^'i■^V.'^■^'^ < "\' 



Welcome to the wVld of -.operations research! 



The":Operations Research (OR.) Approach > • 

Operations research as a. scientific investigation arid eva- . 
luatvon tool, views' the milieu of deci.sioning as a criterion- 
referenced choice b&tween. stated ailternati ves. The term 
_ "operations research" is itself a generic. label for several 
..^ctual tools, and states that .a.'decision situation can be modeled 
(simulated) mathematically . The multiple aljef natives model , " • 
emfJloys a parti cular;.subset|:^f the OR approapti^ called binary 
integer programming , ..which ^ svstpm«; nf Qimii.ltanon,.g 

linear inequalities.,(roughly equivalent'to high school inter- 
mediate algebra). . ■ ' , 

' '. ' > • . ' ' ' 

Via an algebraic representation of the specif ic decisioning 
. framework,- where each criterion is rep^sented as a linear irie- 
. quality (the indepertdent.^vari.ables as 'the programs^ and the 
• v dependent variables as the total system' impact), a value of '1' 
(i.e. -to choose )^ or '0' (.i.e. to not choose ) can be as,si,gned to 
each of the independent vai^iabjes (alternative jsrograms). Tfie 
best mvx of I's'and Q's (across all multiple alternatives) is the 
most optimal solution set. Thus if Program i = '1.,', Program 2 = 
'0' and Program 3 = '1' '(of only three program. alternatives)' the 
decision is to fund programs 'I'- and '3' ;;,artd. therefore not fund, 
•program '2'. This is .the^basi's for 'the multiple 'alternatives 
modeTihg of a fiscal roll-back situation. 



■ 5 ■(* 
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SIMULATION M0DE1.ING. WMHIN A CRITERION- IMPACT DESIGN 



The thedhstical mat^hematlciaff would *hat in the situation 
of fiscal ^oil-backs and .the use of multiple alternatives 
modeMng in •performing such d^isionlng ^- Wie need to determine 
roJl-backs is th^ necessary conditilSii and the utility of the MAM 
technique the sufficieat condition ,^- fhr the existence of the 
mifitiple aUernati|7es modeling fechqique. However, the total uti- 



lity of thjs/model ^.e^tSp'Ss b^yo^nd'^the oti lity^to provjde 
decision-makers with concrete degisions based^ppn a criterion- 
impact, design. " , * ' # " 

Consider the abilfty (of the decision-rnaker) to test various 

hypotheses as to how certain groups of alternatives woul^ impact 

' ■ , . tip ■ 

the system. Consider further the ability to veiry the system's 
parameters (needs, goals, demands, etc.^ and observe the dif-" 
ferehces (if any) of^' programs selected for funding, based upon 
the newly modified constraints. Such abilities suggest a setting 
in which the decision-maker can adcurate^ly (validly and reliably) 
model a system which may not yet exist . It is -the tripartite 
ability to represent; a^system,> experimen1: ^ \yi th-al ternati ves , 
(programs funded 'and/or criteria utiliz'eci) and predict with ^some 
certainty the res'ults- of alternatives actions; Thjs is opera- 
tional modeling in the "crystal ball" setting,, or simulation ; 

The reader may. be musing, "True, but so much of^ the con- 
fidence placed in the results of such a simulat'ive TnS^el ifilist ? 
itself be based, upon the assumptiofi that the'model indeed 
•models' the actual operational setting, both. validly and ^ 
reliably..;* ObViousl34, indisputable. Yet, all of educational 
research was at one tipie (if not sti 1 1 ) ,held . to be non-uti lizable 
due to the inability^tq control all rmtigating and extenuating » 
forces which coivoluted curricuUr learning^ management style and 



teacher attitude findings, ad nauseaum. Recalling that the 
multiple alternatives modeling technique (as In other simulative 
frjameworks) seeks only to. ass1st-^:the. pol Icy analyst's 

;;un'derstandlhg of data.ttf be used -In the actual decision-making 
(regardless of how), we feel confident in saying, "Try it, youMl 
like it. In today's educational ' climate, where experimeQtation 
with policy is often times both , involved and hazardous to one's 
professional health, the MAM framework can with obvious effort 
and diligence uncover the projective relationships between 
program alternatives, criteripn Impact to the system, and 

'.budgeting constraints. ' ' 

■ ' • .. , ■ > 

Monitoring System Impact of Selected. Alternatives 

A final note must be made for fiscal modeling under 
implementation ; that is, what to do when the choices have been 
made, and all decisions are 'goV, 'Borrowing' as we educational 
systems' planners d?>'dl from the electronic engineers during the 
late 1960's, the issueXgf systemic cyberneticism once again beco- 
mes useful. Cybernetic qualities of any modeling stragegy simply 
refers to a careful monitorlzation of the real system under 
implementatioh, as you pu't'\your ^fiscal roll-back decisions 'into 
effect. Now, and heretofore unforseen consequences, impact, 
criterion-references and measurement techniques may be 
discovered, which can be integrated within the original modeling 
framework; that is,' as a, new linear constraint equation or 
inequality. 

The utility of fine-tuning a .decisioning model for more 
accurate future use is certainly moot. As ,tn multivariate 
regression, the model developer may have to try new criteria 
constraints (variablg^s) to associate their variance patterns with 
the decis'ions modeled, and the resulting impact(s) of the 
decisions made. . - 



CHAPTER 02 

• .. ' ■■ 

■ THE CONTEXTUAL STAGE 

■> 



[ Modeling Budgetary Roll-Backs for 
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INTRODUCTION TO MAA MODELING FOR BUDGETARY ROLL-BACKS 

This(chapter explores the philosophical rationale underlying 
the model\(ig of fiscal alternatives in response to budgeting 
cut-backs; and provides a foundation for viewing the program 
funding/allocation decision as either an 'allocation' question 
(i-.e. giving to) or a ,^ deal location* question (I.e. taking away). 
To illustrate the rudiments of decision modeling, the basic 
trends of the traditional cost/benefit model are defined and 
discussed, especially with regard to applying multiple, competing 
criterion measures across multiple (potential) alternatives 
solutions. The four main criterion foci of eff^tiveness, 
efficiency, satisfaction and expenditures are discussed relative 
to multiple decision:;, evaluation; and the ideas of pi^ference and 
trade-off (compromise) necessitated by the existence of multiple 
criteria in competition with one another are summarized. 
Finally, the application of operations research techniques as a 
tool for evaluating potential alternatives is presented for the 
reader's understanding. 

Chapter two prepares the reader for the technical discussion 
(to follow in Chapter four) regarding the actual construction of 
the^multiple alternatives model (MAM), through the development of 
a MAM-orientation in a fiscal budgeting (allocation, etc.) 
situation. Thus this development is situation-specific (to. 
fiscal management) and will hopefully facilitate the . . ^ 
''understanding of the MAW decisioning context. Jhis parallel 
theoretical-application discussion will hopefully allow the more 
discerning reader to view the wide-range of appl ication(s) 
available to^the multiple alternatives design,. The reference 
bibliography at the conclusion of this text will allow the 
masochistically-inclined reader the ability to read more in the 
subject of fiscal budgeting and decision-framework for analyzing 
allocation strategies, ' . 

. . So 
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: j^; PHILOSOPHICAL FOUNDATION FOR FISCAL MODELING 

Fiscal modeling refers- to the studying of an environment in 
which some decisions are required concerning funding allocations 
and expenditure control. Practitioners assume that allocation by 
itself is an automatif interval-control for expenditure. 
Forgotten (or consctoiisly misplaced) is the notion which goes ' 
beyond the question of "how much spent, totally", to the more 
provocative and accountable inquiry,; "h6w. much spent, how (in 
which ways), and where , totally". .-T^^^^^ fn 
order to fund certain programs in; Tieii of; ti^ 
deserving) prograjms necessitates that^the^^ 

(allocator) understands what monies wi 11 be^^^i^ent where , "libw and 

why ; and in addition the impact that such expendl'liure j^^^ 

upon the total (e.g. district) program in philosophy as a >hbT^^^^^^ 

To understand such impact (both validly and reliably) and to be 

able to make the decision(s) required, certain requirements are 

mandated. 

• •* • ^ \ ■ ' ^ ' 

First, an obvious need exists to define, develop and measure 
various criterion-variables in order to be able to compare the 
alternatives; and measure the impact of their funding versus 
non-funding to the system as a whole. For example, women's ath- 
letic programs in higher education have been highly subsidized 6n 
some campuses by income from the men's collegiate-varsity sports 
programs, and from other specially ear-marked funds out of the 
general student-programs administration budget. As budgeting 
cutbacks become a fiscal reality in higher education, ^nd 
increased costs aggravate the existence of less monies, the 
women's sports program becomes a likely candidate. for 'cutback' 
or complete defunding (cut-off). A sample of the impact-criteria 
required by this decisioning situation might be noted as: 

' . I 
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(1) rneasures(s) of total savings, dellneatediintici V^^^ 
expenditure (object codes), so that the wpi^hi'df . e^ch ^ 
'savings* (or *expenditure'\area is known;!V:;//'?J^^ 

(2) rneasure(s) of total impact to the prevaiitrig^^ 
losophy of equal opportunity, equity ancjf okiff^ij^ 
action; and - :/•'• 

(3) measure(s) of relative worth in retainiTig;'iarV^^ 
tinuing this program, compared to othefr^vp**!^ 
(alternatives) which could provide eqij^T;^^^^^^^ 
(eg. campus and grounds maintenance^ /s^cU^^^^^^ 
("bone-head") lower-division courses';"''\.^^^ 

• ■ ■ / . ■'■'■'^ :^ '■ -'^ ^ ■ 

It should be clear that two factors^^^^^ 

MAM-decision. First, the need .exists tOirn^^^ 
:•: impact to the system, whi la i^i nimiii ng any/JlB*ga^irve^'b^^^^ 
i:SeG;ohc^^^^^ ve (tjhofUgH: 

: Mn^icys^ cdiieGtipn of -cri t^eri^a t'hr'ough; Whi^H 'td.^ 

positive as well as negiitive tmpaGt./ In-)!*^ei1it^^ 

(itself) a goal of the models bu.tlder: to qtiTl^ 

kind(s) and most type(s):Of-Griteria in v6rder^\'ta^^^^^ 

results.- ' ^ , - ■ ■ 

Maximizing Program' Goals Within Budgetary :.t1^^^^ 

Fundi ng , cutbacks relate^ both to.. the ^ speci^^^^ program 
(e.g. student acti vi ti es;, gifted. eduie;ati.ony " and 
to the more gen eric definition .Hn. whith^ all 
programs becomes, the 'program^ 'of a:^.iiigh^er^^ same 
di stri ct program)^ In :maximi2i%. th^;^^ nimfzi ng the ' : 

VeviT, the depsion-maker .rn of 
program is being ref erenced. -.Gbyiously;^^ related to 

the co-curricular portian j of ra' studen^^^^^^^^^^ 




'^to maximize the quantity and extent of each student's 
v.,_ participation within co-curricu1„ar activities," 

may have, greater weight to the SBG/ASB advisor than to. the prin-- 
cipal who needs a higher funding level for a 'back-to-basics' 
remedial curriculum. 

At either level, the focus is identical: ' 

"to maximize program goals within ^budgetary limitations", 

while mihimizing the impact of any budgetary cutb-ack'-^de^ to 
the system as a whole. Fiscal modeling thus takes on the 
appearance of a system of compromise — that which is possible 
versus that which is desired. By juxtapositioning maximal bene- ~ 
fit against minimal harm, -ealch^^ and 
" importance" become^ read 1.1 y--^^^^ and avallab1-i^If<ir^-^^^ 

• jDarative evaluation; ' ' 

Partial Defundinq v. Selective Deallocation 

Parallel to the discussion on maximizing program goals, 
(destred. outcomes) *wi thin established budgetary limitations is 
the economic notion of a 'breakreven point ', ^ Often times, the 
^educational decision-maker announces a cut-back decision" on a 
.•percent (or percentage) decrease in allocations. The program 
y^c|airs are advised to "do their best with less'^, often without 
:.i^lections upon whether the resulting limits placed upon pro,gram 
;(|pls can be realistically achieved. At some point (the 
;^|bj^eak-even point') reduced expenditure (reduced funding) results 
|i:li -program performance occuring below acceptable program goals; 

opens a forum for discontinuing that program's operation 



which would then result in the savings of the total potential 
expenditure* The decision-maker must take into account, however, 
the potential of negative outcome to the system; and thereby spn- 
^ider an increase of funding to that particular program, with 
commensurate decrease to another program(s), ' 

„ Clearly, this decision process es complex. Not only must all 
combinations and jJermutationsr'of the programs. • being- compared 
(multiple alternatives) ■ be analyzed, but 'the'system impact of 
each combinatorial permutate^-^rnusj/also be assessed across all 
criteria. (Sounds frightening, ' does it not?) This line of 
thd'ught is"- further aggravated by the aforementioned notion of , 
"multiple funding levels" per program. Yet as hopelessly, ridjcu;-^ 
loiis as it may appear, the decision must be, made --'-and is feein^^^^^ 
J^made in every funding cycle of every district. ■ ■ ' ^ ^ 

Two avenues of approach. ^to the allocation decision .'may be 
made. First, reduced fundiiig allocation is permissable; if and ..^ 
only if .'the resulting reduced allocation does not "significa^ 
(or magnit.udinallyl^^^^^^ both specif ic and generic progr^n 

goals, below ^some.!:^ level . That : isi' . 

fund a prqgr"am;that cannot -fu If tjl . its program goals/at a re^^^ 
expenditure level? The second approach however is a more direc^^^^^^^^ 
maneuver than the parti al defunding approach, ^and can best *be^^;-:^: i 
described as selective deaTlocation. ■ 

Consider the various- multiple alternatives as specific 
programs whose collective outcomes form the district's generic 
program orientation. Further, consider that some of the programs 
are modeled at "full-funding", some at a "minimal ly-acceptable 
level", and others at some discrete point. in between. The deci- 
sion now becomes to fund (allocate) at full or acceptable levels, 
or not at all. This focus upon selective deallocation is of the 
utmost importance, in order to provide a control for regulatory 
accountability to the decisioning framework. 

: . ■ ■ 5^ ^ 

i , ■ ■ ■ 
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Funding Allocation v. Regulatory Accountability 
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The virtue, "better to give than" to receive" cannot be 
applied to fiscal allocation cutbacks during' budgeting crises. 
As was said in the preceding sub-topic, -^^some 'distinguishable 
point must be defined beyond which a cut-back decision automati- 
cally becomes a deallocate ("cut-back") decision. Such decision- 
^ making-must come from the generic-program administrator; the 
■specific-program chair is not likely'to. voluntarily offer such 
suggestions. But since it is true, that "it is easier" to gi-ve 
than to account for", the necessity for some form of regulatory 
accountability is obvious. .,c ' . ■ ■ 

The major concern in thi^^regard^is, of a volatile,WRpliitical 
nature;;; The decision-m;ak^ must take determining 
the level -of -a^^^ and, iiipreover operational ize the 

stance.Jthat Va^ the pVogram wi 11 be deallo- 

•cated instead ;0f partially defunded. It ^s the. opinion of the 
authors that all, fiscal roll-back decisions be made under a 
discrete deallocation philosophy, rather than a progcgssi^ par- 
tial defunding scheme. Such a structured, 'disciplined approach 
is more^ than offset by .the enhanced accountability to .the modeled 
fiscal system. ■ ■ , 



Full Systems', Orientation. To Input 

A model of a fiscal system, assisting the decision-making 
framework for selecting-programs to be funded or defunded 
(rolled-back) is. only as reliable as its ability to simulate that 
system. Reminiscent of the. days of systems 'pi anning, organiza- . 
tional development and participative management/ a fiscal 
system's model must so accurately simulate the original 
environment, th^at^T^^Jj^^^ to the real ' 
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system is also influential to the fiscal model (i.e. vali.dly - 
modeled). Furthermore, output from the systems* model due tcf 
modification of those criterion-variables explaining 
(constraining) the simulated framework, must also reflect the: 
changes expected to the real system (i.e. rel iably mqdeled) . 
Such a one-to-one correspondence between reality' and sim^ulated- 
model requires a full systems * orientation to input. 

Input to arty model simulation refers generally to 'the effect 
imposed upon the model by'th^^^ < / 

exemplify the.^eal system; such criterion-referenced measures are 
known as constraints. .. The utility of full- systems* 'constraints 

•in /accurately consistently-modeling reality' is witnessed nn 
three areas. First , the real system is control,led- by th^main 

^and .interactive effects of input from innumerable sources,- b'oth 
internal and external to the System. In the multiple alter-* 
natives context, such sources are modeled vialfhe use of multiple 
competing .criteria. Although certain sources may be more : V 

/influential (i.e. weiglited) than^others, nevertheless no s'ingle . 
inpiit (effect) exists in isolation-from the cp-related effects 
(inputs) from other sources. , i' - y 

Secondly, the source of multiple criteria; may itself come 
from multiple sources throughout the system. For example, in 
deciding upon a certain curricular program for Implementation, a 
reasonable criterion measure would be .the extent of perceived 
effectiveness in instituting the designed learning change. Such 
perceived change however might be different for each individual 
subgroup: teachers, administrators, students and parents. 
Although a measure of 'learning affectiveness* is desireable, 
necessary also w'ill be the modeling of a decisioning process 
where each of four sources are modeled independently (though 
simultaneously). If the model were to use only a si'ngle 
constraint to input 'a composite measure of effectiveness, the 
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■ or lgin'aT*^ariHnce ..between the five groups would be lost; and the 
system 3nac^at,eT.y'mpclel-' the environment^ surrounding the. 
cjRcisi on;-'--^ ■.>•..■; . ,. . ^ " ' 

£ina^', afu^l 'sysfern'i-qri^ntat^ 
^0 as^ tcr^l low, an. ability ta Gompare'inputed criterion measures 
. (constraYnts') acro$$ the :alternative programs. Only, then can an 
adequate "consensus" model be developed'to portray these system 
• sources of impact. , V ■ * ' , 



• 'f 

-,<r>, ... 



\ 



PROGRAM BUDGETING FOR; AN ALLOCATION/DEALLOCATION; . ' 
•.^^^ . FISCAL STRATEGY ^ 

:. This reader wflT recaVV that a decision for allocation or 
deallocation of funding requires a^ discrete budgeting-level 
f Vamewprk^ VBy. discrete/^^^w^ mean, that ',if a ^particular program can 
satisfactorily accomplish an ^cce of its goals with 

reduced funding, the specific Tevel -of reduced funding -must ''be 
ident1fi.ed\and defined for; that pro:grami^> In this way, .a multiple 
alternatives, deci^ionjngvm^^ for evaluating programs for fiscal 
roll-back can also--assess:' a.Jimi^^ number of discrete levels of • 
funding for dny parti ^ program , 'A' "^at . 

f ulT funding exhibits;: varno perfqjrjnance on .such,' : 

pertinent 'criteria;a^^^^^^ efficiency, as well. as - i'- 

'expenditure level a certain . part of A > 
could be .orriitted fi^bm prograni implementation, without "s:ignif i- 
cantly compromising ..A's, worth,' then It 'is reasonable to evaluate 
'AX'- along With A as two entirely .separate feasible alternatives. 
That is, *AX' will also exhibit its own measures of effectiveness 
and efficiency, A/vith. a- r^ measure for required 

expenditure^leyel. / ■ 

A note of caution and cl.arific^^^ here. The ' 

authors, accepting .the discrete level of. fun^ ^i.n, modeling " l^^ 
funding differences^ thereby rej'ect the closely rel ated Idea of . 
partial funding via percent, reducti oh. It is impossible to 
ascertain the effect upon a. program .'of an" intended 12 percent cut"v!!4C 
in allocation, unless the dollars associated with, the IZ percent 
are identified specifically within the program. The act of o 
r "di voting-Lip" (sic), the reduction across all shares equally, is 
both unreasonable and. irrational "^(but we choose not to overstate. 
our case) . * . ' ' • \ -^^ 'r/ 



■ • The deG,i.s.1ontng,framework surroundingfun^ and\ 
revenuealldcation has often; iraditignaliy 

concept of system-building. Under this paradigm, -no- program? .' ■ 
. . exist a priori, ; and therefore all potent i 'a] programs compete .1 \ 
'. (though unequally) for some^roraition of the total.' avai 1 able 
.. budget. Educationr became very enamoured, with this concept of- ' - ^ 
• budgeting, referred; to as zero-base budgeting; and- many units' 
used the concept during the early I970's. - ' . / 

. The philosophical elegance .of a 'zero-base', rjiodel is 

interesting i.f 'not intoxpcatiog: Requiring each 'program to revi- 
;, sit its 'roots,' and thus' "'stanc!'' the^ha,T-leng6 frora^other .com- 
, petitive programs as they support ^heir Claims. |or^^ev6ji * 
. increasing levels of projected' expe'nditure,. Others believe th^ ^ 
. ? the elegance of the;_ model ends with: the statement of its * 
philosophy. " ■ . ' ^' " • 



. Seliecting programs for fuhding (that is, system bui.lding) can ~. 
^ also be viewed as an assessment-procedure for •evaluating certain- ° 
/ alternative programs 'tO: be added ta ^n. already -on-going system, ^ ' 
..■ and therebi^ .provide some degree"^qf enhancement to the system' s 
mission. crises however,, the question is • 
(nDrma11y[:> what .dg ;Me ;tut?; - not, what dp we 'add? /For this, 
^reason, , the •modeling (^r^;a f^^^^ dec/isiori-making prq^: 

cess pan easirjy .assum|'^ the operational' characteristics pf . the i 
. ■ ^^wo'^'^J^fiat is, basec(,Vupon:.a certain reduced experi- 

■ diture; bu^dg^aK w^g^■£^^ ', 
be :i',iln'<^..e^^^^^^^ cupre'ntly bperahng' programs^ • 



' :• ■ ■ • 



Revising the -Fiscal Program System ... 

An alternative to the philosophy of bui.lding anew the system' 

in order to indirectly determtne cutbacks, is the- idea, of: 

. ■*/ " • 

given the current ""system t)f operating programs and^ their 
, impact/ effect upon the system ^s whole, what progracns* 

can be directly selected for rol 1-back- based upon their 
modeled performance criteria? a ^ ^ 

Through the philosophical stance of revision, , the overall objec- 
tive becomes to choose programs for deallocation while minimizing 
a decreased satisfaction of required/desired system goals, etc. 
In the case of fiscal roll-backs, a rev-tsion approach is the pre- 

f erred procedure, though in -a modified sense. 

- ^ ■ ■ . 

Since many educational systems are -so large as to have 
hundreds of model-related programs, it would be very time- 
consuming to require the modeling of entire sysftems. An alter- 
native is to model only those alternatives (programs, potential 1^ 
available for cut-back); and to choose from this list of • 
'feasible' expendatble programs for^sqlving the fiscal roll-back 
issue. V 

From a modeling protocol, the role of constraints in guiding 
the fiscal roll-b.ack decision may be seen as: m-i^imizing the 
loss of the contribution to total systems effectiveness and ^ 
efficiency; while concurrently maximizing the expenditure dif- 
ferential which is destined for roll-back. 




Benefits- of Itemized Budgetfnq artd Delineated Proqf*aiTmiinq 

« dirS cussed earlier, the lise of delineated |^gramming in 

the fprni^^iOTfTnultiple-iprogram ''3^ersJ!bns»;^i\h different-^ projections 

% of discrete' levels of funding, %an b^very beneficial in giodeling 
fiscal systems for /oil-backs. It Wci^s ajj|o stated'^h at' knowledge^ 
of the level pf required funding (as a.compos^ts measure) was not 

' as useful as a differentiation of the requi reallocation Into,, 
specific delineated object ar-eas -of expendftifrey *, 

>' The typical educational budget is' grouped .into a series of ' 
expenditure areas (ca^lled objeq|;§t^ which pertain to such foci as 
salaries,, benefits, supplies, and tnaterijls, equipmerrt and capital 
outlay. In a rolT-backcdet1%ion, it* is reasonable that ^he 
decision-maker m^y desire to (;pnstrain somj? area (object) of 
furtding greater than another. • For e^xample, the reduction in the 
- amqwnt of a floated bonjj i3sue^.-m'ay require backs, siTch thaj" 
the ^capital expenditure objectVijiust be more' titeverely . ' 

eonstrai(09d than other areas (Jf object expend itlii?^e^ Obviously^' 
the administrator -cannot allow programs to be implefnented if a., 
capital Q|^lay is mandatory, to the success qf'^these (Orograms, 
with no capital mo^es available. 5, o 

" " <^ . . . ■■ ■ \. 

Often times the decjjgion-maker may jjish' to ^egre^te, those 
programs which exceed the* ' average expenditure ffevei' from the^'*^ 
remaining programs for more .detai led scrut^Tiy. Such %t- eva-.^ 
luation could easily be a useful strategy immedftately preceding a 
full fiscal , study of . the current operating system. .J^nally, the 
impact upon the system of^ proposed rol>l-&acks determin^^y a - 



multiple alternatives modeling techniq-Lie,^^/can only be vjewed vie\;. 
the Individual expenditure categories if* and only i^ > the indiv-i- 
dual categories were original ly modeled. v 
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Testing for^Strengths, Weaknesses and llesponsiveness to Stated 
Needs ' 

Prior to our eminent discussion of the cost/benefit modeling 
framework as a historical forerunner to the more powerful opera- 
tions research technique we call the multiple alternatives model , 
it is advisable that the' rationale underlying otfr preceding com- 
ments be. reiterated; 

Fiscal funding crises^ require (normally) some degre^ of 
expenditure cut-back; it. has been the theme of this paper that 
:.such dreclsiotis should be program-whol IsticSl ly oriented as com- ^ 
pared to leveling »a certain,, "equitable" percentage share , across 
all programs. In other words, it may be more rational to discon- 
tinue an entire specific program, as compared to under-funding 
several of the generic program'' s specific entries. To opera- ^ 
tionalize this phtlosophy, all programs are viewed as^multiple , 
alternatives to a fiscal roll-back decision; and measured cri- 
teria are used (as contraints) to evaluate all potential^ com- 
'binations .gind permutations of these alternatives, to determine^ 

how many programs , must be 'cut; 'and which ones. Discrete -levels 

1^ — 

of funding in order to determine various delineated programming 

^'c^- ■ ' '/ ■ - 

fllternatiyes has been discussed as a recommended procedure. 

v The rationale Ifl the preceding sections has been presented in 
order to introduce a particular philosophy; and that philosophy 
' ref lects./thre necessityirof testitng for the comparable strengths 
• ^nd weaknesses between ana among program deallocation 
alternatives; and to specif iQ'&lly determine (understand) each ^ 
prggram's (or group of programs) responsiveness to expressed « , 
needs' ofi^ the problem origiitally intended for remediation. In 
short, to know what a program is dqing and how , r.and to be able to 
st**ate why that particular program (setectecf via '-evaluation 

" I' it 

^modeling) was 'rblled-back' , Such are the ingredients of a 
'dalA-based, accountable decision. - 



TRADITIONAL MODELING VIA COST ANALYTICAL DESIGN 



The plethora of cost analytical frameworks has focused mainly 
.upon four specific evaluative,.or modeling^ designs: cost-benefit, 
cost-effectiveness, cost-uti lity and (though hardly an analytical 
framework, per se) cost-feasibility analyses. Some of these 
modejs support the use gf multiple criteria related to a single 
focus, whil§ other models prefer a singular criteribn formed via 
the composite of multiple foci; but all models agre^pon at 
least one postulate: 

The analysis (and subsequent selection) of an alternative 
course of action from among multiple alternatives; subject to 
the evaluation of each of the alternatives across multiple 
(or ^singular) criteria, which are purported to measure the 
alternative's impact upon the system (of decisioning) being 
modeled; 

and such that: 



(1) positive effects to the system are^maximized; 
- (2) negative effects (as by-products) are minimized; and 
(3) neutral effects . (as desireable) are maintained at the 
central tendency of measured impact. 

To accomplish' this end-result, cost-analysis modeling has deve- 

hi 

loped into a science of graphic displays, -measurement schemes, 
and statistical overlays. To date, however, the serious shcj^t- 
cOming*of many of the cost-analytical designs, has been the ^ 
rnoders inability^to adequately control for interactive effects 
between (and among) <:riter1a for any partic^ular alternative^bei-ng ^: 
evaluated; and an inherent unreliability to systemi'cal ly evaluate 
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Figure 3. Standard Decision Matrix for Criter ion-Ref erenceS 
Analysis of Multiple Alternatives. 



Multiple alternatives oeing analyzed. 
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a>* multiple alternatives solution (where ,the selection -of more- 
than one alternative is necessary to adequately- satisfy, the^^ 
required demandsfneeds of / the system, being modeled)'. . Before " • 
solving this difficult problem of multiple ^solutions' aiiross 
multiple criteria, the reader must first "grasp the 4Ti.orp. tr adi- . 
tional aspects of cost-analysis desigir( and. mpdel ing. : / 



- Application of a Decisioning Matrix ; - ' ^ , * . . . 

The choice of a solution from among mul tiple/altefnati.ves, . 
.via the evaluation of each alternative across multiple criteria; 
is easily viewed in a decisioning matrix format <F^gure.'3>. : • , " 
With each column representing the values of stated criteria for a 
particular alternative, a m X ,n matrix is formed; consisting of 
m-criteria (measures) across each of n- alternatives (defined); 
And as a 5x8 matrix yields (5)(8)=40 cells, so does a n - 
matrix yield (m)(n)rmn measures for evaluation. It rem^aitiS' these 
mn measures' which will then be utilized by the decision-makfer/.to 
judge which alternative actlon(s) is (are) the 'best' "sol u.tf'df|'j(s) 
to the problem being modeled. , -''-y 

The decisi oning matrix provides' a useful formulation' for |h(|- 
eventual modeling of the fiscal rollrback cbntext.' Defining each 
of the various alternatiVes (Aj, j=l, .. .,8) :di.splaye(/'^in. Figtir^e^ 
1 as potential programs to be r\)l l«d-back in a dealTocation , • 
decision, . the objective becomes: to 'select ^that particular : • 
alternative (Aj); which best exemp^ifies^:'tli^'.state5|^p^^ 
used to make the dealioi^ation dec^lsion; .aii/^hich; . 
bal ances the budg.et . In rea^^':^1; <^^^ie the • expe'rl ehced prac- 
ti ti oner real i zes that more ^ns'a sf ngH'^'aTternVt^i ve- pf o grate rfav 
require rol 1-back if ^the crtter^ortT; object}^ f d bevmlt. ■. F.pr^ 

the purposes' of Instruction Si J f?lus%r^f on' o^f^ 
however,, only a slngle-alterhatvye context wVlJibe ijlustrWed.' ' 



■'5 



The multipleT-alternatives context will be discussed in a later 
section. . (Your patience will iie re}j^drded.)/ :^ ■ ^ , 



For each alternative Aj, -then, 'there; exists a, s^^^ 

of criterion measures, a^j (i = l.. ..,5), refl^^ 

."nature(s)" of the alternative as measured by each of : . 

i-criteria.: In ;the ''s.ejfectito^ single alternartive/ the 
; problem involves; vCorrtp^^ measures in a reliable 

fas-hionv^s^ each of the particular • 

e^lieir^nat^^ being mtfdelQtI i s:\und and thus a reasonable,;:;; 

rati 6^ inay be. made ( or at • 1 east , . v ■ 

l^"^ ojshi sti^^c^^^^^ at" ) i ; To raccompl i sh thi s . analyst s of , , 

.the criterion i^pipact .via each;^[lterriati ve upon the system, ;a spe- 
"/ ■■ci.fio^procedur^''must be devised;- ; V; .. : ;; : - ' i; ; ^ : " 

■.,-./... -r-^ - ■; : ■/ •■^■v...-;. ; -V Ar,:;v v -i' '-Mi.^-- ^ - 



'^V . ThQ Cbmposite Vari afa1 e .f^^'nki ng ' ( CVR ) ProCjedLn^a''^ 



rfe1i.able;:appLl:katiqri^ of^a^^ the .evaluation 

^ikH^n^^it^ access mul-^i pie 'competing criteria,- must 

^saLtiis^:;a^ , > /: r ' - - 

( 1 y cOmpa^ of Iniil jbi pi e ||ri teri a^ora 

; V • ead^ ,a sH ng 1 e c omp 6 s i^t e v a1 u e ; ^0 

; ; r e^resentihg earn. iMi^^ ar ^al ternat i ve ,be computed . M 

G^-' The re WtPii'tkt value req^^ires^l 

" ^ of the -avMi^h^^^^ rescaled tg a 

.. ^ . ; common "measui^etnent fqrmat; , #ath.4W terms of units^ (e.g:-^" 
, : , , / ;do^larSr s^iJ?^^ of pupils) as well as 

. '^ ■ -^cATlng (:nd^ ratio) — that is, 

. : ■;. . -50 tKat -. apples?. Cxin :be^cpmpaf::6d ^ oranges ; . 




(3) The evaluation of the impact upon, the system frorti the 
criteria being used, requires a method for analysing the / 
criterion impact across all alternatives (as wel'l as the^ 
value of each alternative across all criteria) and 
finally, 

■ ^ ■ .:• * 

(4) The realistic modeling of a decisioning context, • 
requires the ability to "weigh" the various criteria^ ; 
being utilized, and thereby vary the. relative importance 
of the criterion effect upon the decision(s) being made 
-(alternative choices), * • . • 

The Composite Variable Ranking (CVR). model has been designed 
to specifically address these four requirements. After the ini- 
tial measurement of the criterion variables has been accomplished 
(e.g. cost of programs parts in 'dollars'; space requirements in 
'square feet* or 'number of. rooms '; personnel in total 'PTEs ';. ' 
etc.), the normalized T-scores of the relative raw measures are 
computed for each criterion variable (across the range of ^ 
measured al ter natives )..^, That is, : 

. 'A * 

> . • ■ • ' ' ■ ' '.' ' . ■ ■• • ^ ■ ■ ■■ . , 

• ij' J = ls^>n y.^ T^j, j = l,...,n 

for each criterion i=l,...,m., This conversion replaces all raw 
measures (square feet, dQ^llars, etc.) with Its associated distri- 
butional THiormal. .T-nbrmals by definition have a mean. of 50.0 
and a standard deviation of 10.0.. 'Thus, a facility-space measure 
of 2560 square feet for pro-am. alter G has a T-measure.of 

50.0 if 25J50 square feet' is also the distributional mean across 
all programs for space requirements. Likewise, a personnel 
requirement of 12 teacher aides^: has the T-yalue of 50;o if 12 
(TAs) 1^ the distributional mean across alT programs. , 



The composite variable ranking procedure summates each 
column's row entries (that is, adds the criterion T-measures for 
each alternatiye), producing a single composite measure per each 
alternative being evaluated. Th^e T-normal sums can then .be 
ranked such that their ordinal ly-comparative importance to the ^ 
decision be recognized. ' ^ . 

Likewise, the row^/can be summed (i .e. , adding across each row Vs 
column entries), to 'Understand the relative impact of each criterion 
across all alternatives within the system. Finally, standard 
weighting practices can be applied to the criterion T-normals 
normalization, of course) before the summation of the column vector ' 
entries/ 

The CVR model ing technique, is an excellent field-tested and 
validated technique for performing most decision an^ilyses ; , 
involving decison. matrixes. Moreover, the CVR approach is well- 
defined and easily constructed for a fiscal program alternative's 
setting. The technique is not without its ^inher^nt inadequacies, 
however, centering mostly around. its predominant reliance upon, 
both a singular alternative context and main-effects modeling. • 



• Main Effects Modgfling ^ 



In an earlier section of this report, the issue of multiple 
alternatives modeling (MAM) was discussed in the context of a. 
solution requiring not just one alternative, but rath$r a finite 
group of alternatives (referred to as the "alternatives-mix^set". , 
in the first topic of the next section). If a decisioning model 
purport? to truly simulate a teal situation, then the model must 
be able to compare^'roups of ^alternatives against other groups of 
alternatives, tjfj^lizing the criterion nfieasures which have been 
selected to simulate. the impact of the alternatives upon the 
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Figure .2: Representattpri "bf the Compos Ue Variable Ranking 
(CVR) FormuTlzation for Main-Eff^^^ 
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system being rfrbdeled. THis is the main operational difference 
between, singular and .multiple alternatives modeling — that 
several alternatives may. require operatdnalization to, satisfac- 
torily remediate the identified problem. 

Main effects model ing^ is a cbrreTated idea to the multiple ' 

■ ■ ■ ■ ' ■ * '. . ' > '■ «... 

alternatives context, from the standpoint of the multiple- 
criterion effect via each , evaluated\-al ternatt^^^ Consider the) 
following example. Five alternatives have been, identified as^ 
potential remedial actions for a particular problem^being ' 
modeled. Each alternative was measured across each of three, cri- 
. teria to permit a criterton-refeifenced .e\faluation. (To save 
time, and wear-and-tear on the authors, the transformation to T- 
normals is suspended for this discussion).. The measurement scale 
chosen, was -a 5-point scalV with^ interval of 1 unit, signifying 
low^^benef it (=1). to high benefit (=5).. The simplified .decision . 
matrix is shown^in cFigure 4>. 

^ Note that the column* sums indicate Alternative-C as a clear.' 
.'winner- in this "identify the most beneficial alternative" 
contest. 'However, also - note "that although C's. sumVwas the' 
highest, ^the measure of criteria #2 for C (=2) suggests G 
demonstrates, moderately-lbw performance, benefit on this criterion 
measure. If we approach this .siiriijlated example fr^m a *.muitipTe 
alternatives' vantage, "a likeTy^.s might be the ,incd)r-^ . 

poration^ of both C aiid B into an: alternatives mix-set . M^t6 how • 
B's measure of ;moderately-high: benefit (==4) ^oii /criterion #2 coun- 
te);>^ts C'S;moderatel7-low X=2) value. . Also, note how C must 
then m&ke up for "B's apparent disadvantage regarding criteria #1 
•and #3.. ■ . .■ 

»■ • ' -o . " ' ■ ■ ■ '. 

.. ■ ; . . ; . V ■ . . ■ o ■ ■ 

. /Main-effects' model iiig would have computed -Qiy^lumnar 
summations, and selected the .alternative 'C' as^^^ost- likely 
;So1ution^. It is 'just as likely that in a more c^pl^i^kted ' 




simulation, the decision-maker might not reqognize the criterion 
#2 influence of alternative C. Clearly, this situation is a 
potenti al problem with both singular and multiply. alternatives 
modeling simulations. ; 

'■ ' ' • * *■.'■.' • • ' . • 

Jhe.,solutioir^ is to performvwh "interactive 
effects^*! modeling — controlling not. only for the presence of . 
multiple solutions, but also controlling for the. potenti al of 
sub-optimal criterion measures for a given alternative which may 
be .-masked by the values of other highly-beneficial criterion 
measures. The illustration and application of just, such a model,, 
the Multi'ple Alternatives Model , within a fiscal rollr-baGk con- 

• Of/text is the^subject of this report. 

Generic^.Criteri.a for Competing Alternatives . . ' 

We. have spent a great deal of energy thus far^,. in describing. 
. what to do, how and why ... .but have gingerly maneuvered around' 
^ the question of ^with. what' viz . criteria . Criteria must be. both 
- system-specif ic. 'as wel 1 as al ter,native(s)-specif ic ^ Roughly' / 
translated, criterior> m^sures must reflect both^.the system being 
modeled as well as •the.;a1ternati ve solutions being-devaluated, • 
respectively. Otherwise, the impact to the system' cannot be 
measured, ..ai nee it had not been model ed. fi .e.: -S^l ated) . : • 

^ : the-|^pij^ roll-backs is.no 

. exceptlpr^l^ and analyzed : 

across al:fv|tlternati vest's the, alternatives cjin be : 

.■^alidly compared within a'budgeting context; and the itiipadt to, 
, the system of each alternative (or combinatorial premi^tati on of/ 

• all available .al.terh|j3ves): can/be analyzed. -Finally, criteria' ■, 
... must; be col lectively^; exhaustive of the 'foci ' required to , 

. criterion-modefl the^ decisjoning context,;'' and allow cross^ 

. ' ■■ '■■ ^ '} . ■ ■ 7r ■ ■; ^ ■ ■ ■ n,*.^ ,„ ' ' : 
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comparisons bet^^ea criterion measures, -ia order to check for 
cp.llectiv^ly unacceptable 'impact' values (interactive effects A-; 
modeling). \ , ^ . ; 

Modeling alternatives- witM the fiscal domain represents as. 
..clear\ai.n ..i;lfustration' .0 

multiple-alternatives^ 'decision!^ For traditional 

cost analytical 'stuMes, ^ the generic focus of expenditure rhas 
been the province of eost-benef it analysis. Similarly, ;foci of 
.. effectiveness and satisfaction have remained strong c'ritfir^i on 
.entriesin cist-eff ectiveness and cost-utility analyses^ . ' " . 
. refspecti vely. An additionaT measure- focus of efficiency could' 
find itself In either of the three cost-analysis models, 
, d^^hding upon its source of data (as. is probably true for all of 
the,'i;nitial three criterion .foci )• 

Nevertheless-, these four generic criterion foci (effective- • 
*ess, efficiency, satisfaction and expenditure) are. directly 
applicable to the fiscal modeli^-ng domain: i . r: 

. -/ • '. ■ ■■■ ' ■ ; ' ■ • . . ■ ' D, . ■ ■. ' '■ ■ ■ . '■ • :•. ■ . 

■ ■ ' . ■' '. ■' ■ . / • ■/ ■■ ■ 

^ Effectiveness • ^ '^V^^' - • v/?^|rtj||f 

. I. HOW effective are each of ;the various al tern at^:)^^ 
* * ■/ /programs, in, promoting the: district's generic prbgrani' 
. <v goals? ■ •• . • ■ ■ ; . ■ : 



I? 



2V How effective' are each of the variuous • alternative 
> v ..programs . in optimal Ty reducing the current, problems 

associated with. each of the districts* school * s specif ic 
program goals? \- \ • - ^ .• / 
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Efficiency ■ ■ ' ' '' 

1 .. How ef f 1 ci ent' are each*.of t>ie . var i biis al ternati ve 
'. .: .pnografe in . conducti ng, the required, i rtstPucti onal 
'■••^^programs of ihe^di-strict? ^^-'^ / " ' . 



2. 



Hpjy eff i c^ ent. are each i.bf.-th^ ■ var i qus al ternati ve - ' . 
prQgrams'; in reniedj ating the current- prp.blem{s) . to b6 
solved, within the; di'strict?. ' ^ . 



Satisfactiba --..V ; ^ ■ ''^^ 

, l^^w '^ati^^^^^^^ of the variou^ a} t|rr^^ve• 

; programs %Tn. th§ir executi.6ri,V based' upon ,, the d^^ *' 
■' - ; tionaivdomain?'' of the" administrliton,' te^^^^^ 'student, • 
' , "parent, and school.^ bo^^^^ ° ; .• "''i ' ■ , ' 

. ■■■■ ■ . ■ ■ , ■ ' ■ ' •' ^ ■ ■.■ , ■ ■ . "A ■ • 

2.* How sati sf actory are each of^^jj^ie; yari ous alternative 

. programs in .their remediation of the i^entif ie'd ■ . ' 

; prol5.1em(s,)," ba?ei upon the d^ qf the* 

*' amdi nis^rator, ./teacher,, ^tu'dent^^ and schoo,J 

^: -.boarcir , ; V ■ v ' (' ^ f::-' ' ' V . ' 



y ' ., ■ ^i.v.>.^^--" . •■ 

Expenditure ' '"'^^ ' " - '• - ■' ' ' " ' ' ' 

-i. What -are the; s[||cific object ;Costs ta the^^^^ for 
. '^ each of the \(,ar>ous alfernati.ves; .,a^^^ y 
' •■>' ;s-aving's' if roTTejd-jDack? ^ v: ^ •'■■^ •■ V ■ ' / .■ 

2. What are the costs to the distri'et in . terms ^of l^enef its 
• ;,if the programs continiile? .. " . ■ .' ' 

■ . ■ V,.-',,;- ^ , , ; ; ^ •, , /v^., ;, ■ .,■ \. / 

3. ' What 'are the costV'tp thl; 4istrict i terms of ,-'.Voss ''if' 

"the programs- are roT:led-b|^ 




It. is Ukely (if not strongly suggested) that sever a<l^cri- 
teria (measurement variables) would be identified to adequately^ 
model the rather general idea expressed • above. For example, the^^ 
criterion fogus-^TDf efficiency for a particular set of alter- 
natives curriculum programs might be measured in terms of: 

Amount of time in minutes the program requires for 
instrumentation each week; 

Number of students that could be handled per class 
session (to identify small v. medium v. large group 
sessions); ^nd/or , . 

(3) Pfer,cent of time the" program requires use of a particular 
laboratory or library resource room. 

Expenditure is 'another criterion focus particularly^suscep- 
tible to the 'delineation* of its content. For example,* the 
total, cost of a program is -important; but potentially more impor- 
tant is the program's budget-breakdown by. object expenditure 
(e.g. amount for salaries v. amount required for capital 
improvement)'. <Figure 5> illustrates^the. impact of differen- 
tiated criterion foci upon one traditional decision matrix. 

It may now be trivial to state that each of the fotir sub- 
matrices within the totaT decision matrix could be: i.tself a deci- 
son sub-matrix. Thus the a x n effectiveness-. sub-matrix could be 
executed to determine which alternatives best "'fit* one stated 
■effectiveness criteria. In turn, the bxri. efficiency sffb-matrix 
could be executed for its solution; and f:hen each of the - 
remaining c x^n satisfaction and d x n expenditure sub-matrices 
could be evaluated. SiiGh -a serial procedure would yield .four / 
sets of answers (alternative solutions), which themselves would 
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rigur^.T. Representation of a Generic-Criterion Dscisioning Model for 
Analyzing Multiple Competing Alternatives. 



/ 



Cri terion 



Foci 



*^ ^(Effectiveness Criteria) 



CRIT^ 



EFFEC-1 
EFFEC-2 



CRlf^' ' EFFEC-a 
a 



(Efficiency Criteria) 



CRIT,^, . EFFIC-I 



CRI" 



CRIT 



a+b 



EFFIC-2 r 



EFFIC-b 



Multiole Alternatives 



(Satisfaction Criteria) 

. c; 



.^a^-o+l 



CRIT 



SATIS-l 
SATIS-2 

SATIS-C 



(Expsnijit'jre Criteria) 



.CRIT 



a^-b^-c^fJ 



EXPEN-d' 



a X n suo-matnx 



effectiveness measures 
across al ternati ves . 



b X n.sub-matnx 



effectiveness ..measures 
' across al ternati ves 



c X n sub-matrix 



«) 



satisfaction measures 
across al ternati ves 



d X n "iub-matrix 

nz 



axpencmure incroases 
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require comparison for^a. final solution.. The question arises, "Is 
this really the best, nfd'st" valid (and reliable) process to 

• ^ ■ • 4 ; ■ 

follow?" , : ' 

Hopefully it is alsb^Urty^ial (?) to the reader, that the full 
decision matrix (a+b+c+d)' o^yrt..^ have be^fin evaluated; the 
solutions determined, reflecting those alt^hatives which best 
fit the total eff ectiveness, eff iciency^-^^V^^^ and expen-'' 

diture criterion sets,' s i m u 1 1 an ec u s 1 yi ^Jt^ci - 1 h u s , optimally opera- 
tionalize the preferred interactive e^lf^?tis ' model inq framework as 
previously discussed. ^ 

A Preference/Trade-Off Analytical Framework 

The importance of variable criterion characteristics for a 
given alternative must, be reiterated. Solutions to real-life 
problems are found to be "perfect" only in textbooks, professor.^s 
lecture nbtes, and the 1950's cinema. In reality, all potential 
alternative solutions willbe found to have at least one flaw (if 
not many); and still be the best al ternati ve(s) solution 
avail able. 

In selecting a final alternative as a solution based upon 
.that same alternative's rnerits, the decision-maker also 
(consciously, we hope) accepts, that sajne alternative's lack of 
merit on other less rv1.Huous\:.crite)^ Recall the 

illustration in <Figure '4>; :A1 terljnative C was selected based ^ 
upon\the rrlieritsprf''?n^^ #1 and #3. 'To fill the gap indi- 
cated by criterlia #2,^ alternatives mix-set solution^ 
was sought* withVtHe sJreequent addition of *B' to the solutim 
set. However had we the option to choose a set of solution 
alternatives, would we have retained alternative 'C At this 
level of macroanal^sis, \he answer is probably 'yes'. 



This is the theory, of preference/trade-off in altllrnatives 
modeling singular or- multiple. Alternative C was the fina.1 
choice due-to one preference for high benefit on criteria #1 and 
#3; and a concurrent willingness to trade-off (i.e. accept the ., 
negative) the low benefit effect associated with criterion #2. 
This concept is most important in the understanding of the 
multiple-alternatives/ interactive-effects modeling technique to 
be illustrated in Part II of\his report;' and applied to the 
fiscal roll-back problem in Part III. The main difference bet- 
ween the way the concept of preference/tradeoff has been 
described, and the way in which it is actually applied will be 
evident. Basically, the multiple alternatives model (MAM) will 
define preference/trade-off as a willingess to accept a central- 
ten>dency solution mix-set, where the required impactMs not 
alternative-specific, but rather is mix-set general i;^ed. The * 
measures of central tendency and variablility (distfributionll 
mean and standard deviation) will be applied to a yet-to-be- 
discussed marvelous vector of values* cal lecf the conditional^ 
vector, in order to assume their preference/trade-off^ . ^ 
flexibility. * ^ e 
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SCHOOL CLOSURES AS A' fISCAL BUDGETING DECISION .\y 

' ' '* ^' ' » . 

Few decision modeling situations exist .without ^ome amount- 
of reliance upon selected criteria wh'ich define mbnetary items. 
Indeed, it could be said, that all. decisions involve pot^tial 
changes within expenditure categories "based upon the alternatives 
selected; that is, expendi types will eitl^er increase, decrease or 
remain the same, wh4Je revenues are increasing, decreasing or 
likewise constant. This may seem like 'overstating the obvio'fis 
case', yet many decisionals often seem determined to study thbse ^ 
criterion items concerned with need^ demand and performance— 
and relegate to final stage, a decision as .to whether the selected 
alternatives can be funded or not within the exi^^ng budgtary 
limits; or whether the newly-deci'ded roll-back in-iprbgrams wi 11 
satisfy the reduced expenditure level sought. . ^^ ' /.v ^ 

■a , ^ ' ■ • 

In another sense, concern with the fiscal' portion of probTeni'' ' 
resolution may in fact signal "the reasonableness of a partiq^lar^ V;^^ 
policy decision; and the likel iness' of a particular po]icy alte1^^'V; v 
native in resolving a problem, issue. , In tods^ffs .educational '^-^ 
climate, decreased revenues for instruction, salaries and buildi^§<^^: ' 
maintenance have announced the era for the clp^sing of schftaAs a^l^!^^^^^^ 
a cost savings stratagem. Opponents^ of a school closufli^pr^^ 
attack such an solution alternative as not cost-^eff edtp'^J^^te^ 
of expenditure savings, and not cost-benefit in terms;' o"^ 
decrease in educational quality, the author hasr'explor^ this- S. i^^ 
issue, and presents the followi'ng as an illustration of uti lizing 
fiscal criteria in determining- an initial Vol icy standard i;- prioc 
to developing the various multiple 'implementation alternatives' •i.y 
which would execute the policy^ and its implications. . .< . 



Cost-Benefit and SchooTHosures ( 



A;se1ected alternative is viib'Te' and feasible only insofar- 
■ as 'its' affectiyeness; iiiyremediating a particul ar stated' problem' 
is qonciusiyely .defined (Wholebeni, 1980a). Within a competiag • . 
si tuition about by declining schopT enrol'l- 

ments, :*to%j^^ management: accounta- 

■ \; ^'•''•^ty* spiral in^ solution 

,as'?scliool closure's and attendance zone cbnsol- 
::.«£l^^ reducifiq'.expenditu»*e ' l evels, m reduce or 

ll^^} t^n.^nate, -the negative 'bi^ of Ihe defined problem, but- also 
■W^*o'"!^$« 'Positivfelp^-r^^^ ' ' ' 



.... 



mMM'^^^'^'.^y^^^^ -^of coi^:bene1^t.;4nd^progy^^'^^^^ within the 



0i^yj^sici\'-qf these ,t)ire|'Mssues^cdncei^^^^^^^^^ 

'f*- aspect of ■''educatidnal tonsMmershiij': The depi si on 

. of the p,c|,Jcy; -in^alyst ttj' select schboi, ctbstires as a means of 
-'.abati ng th|^' negati v^ ihfl uiences caused by declining educi|tiohal 

^fe"'"^''^.'^^^ ^^^^^ wi th, it t those 
i^f ppsntii ,^^)^Tt^^ references at remairtirtg: pperational stt^^ in 
'f <-'.,brd€r'; pol icy ;to be a viabte: alternative.' 




ysjcal Many LftvTi zati on -and Related Fiscal Resource Allocation 

- . V* ■ • : . .. ■ . - ■ 

School sites maintain certain fixed costs and/or 'werhead ^ 
rl^^^v^^^^ to" the level of educational- / 

,Qpeif^tion.*^^ch Vequir^^htl heating, air-conditioning. 



^ - l^htihg, maintenance, pi^Jjici.^^^^^^ interest payments ort bonds, 
^7 and grounds maintenance are ttie more immediate issue^tif fiscal 
:^|^nand that are , got fully explained by the degree of building 



■ ■•3 • . * ■ ■ 
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use^X Givert sQnie^^ measure of cost-benefit 

(e.g. Heatings tHel^^^^ .or maintenance 

requ i remeh t per;^ *< '^c an it ait u s ) " prpport i on al 1 y gr eat er .. * 
energy and^ fistaiV V^^^^ operational 



.occupancyw-;^ 



Physical/.uVi.lizatf^^^^ and the costs associated ; 

Wi.th'f utility, are 'the: sa^^^ measure, for criterion generation. 



Irtjdep^ndent - of*, tbeyji^^e^^^ .i s.s.ues of program imp! ementat ion and * 

staffing, theyi^]qiptHj|^a^^ overhead is generally not under- . 

^^stodef-^y^^^^^^t^^ Criterion measures as energy 

: l^jiermsf^R^i;^ :.s instructipnal/Space per 

en«fDlleU;^i6t'4d potential ~ are clouded 



by notions size equals better education", and 

v ':!:Tes$ :*;$;itev'ei^ro1^ less on-site maintenance^ expenditure" 

j Unfort]jria^eTy,''tiud^*^ lack empirical substance. 

* * • Examil^^-'Speplf ical ly the probable bases for these views, . 
'V; It "ha^, been f&uhd>''%^ in a school district of 32 elementary 

V-sc^i^^ 

■ yv^!:;'^^^^^^ of enrollment explains less than 

> * / 13.7/percent of the variance associated 

% ^ ' ' ' . • 

.'/^/'-'ji tf>t^ maintenance expenditure^; and less than 

^"^ -"^KO percent of the variance associated 
/with total utility expenditures; \^ ' 



btJt that: 

site area e;(plai]^[ed 53.-5 percent of associated 
• VV" heating requirement^, 47.5 percent of ijelated 

" , utility expenditures, and 40.6 percent of 
' total service (maip^nance plus repair and 

replacement) costsHwholeben, 1980a). 
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Such findings would suggest, that fixed costs exist at a 
■'relatively qonstant leveT^ regardless of building utilization, 
ft'necessarily follows, that the more such costs ar;e -distributed 
across reduced sites, the lesser the level of total expenditure. 



Curriculum Program Flexibility and l^elated Cost-Benef it Indicators 

Not as easily quantifiable yet no less- reliable are the ^ 
trade-offs associated with specialized curriculum" programs (i.e. 
those requiring speci|al ly trained personnel) and fiscal allotments 

Even if (only) a single student requires specialized assist- 
ance — that assistance must be forthcoming. Special programs 
require special people (normally with more specializjed training), 
and therefore at greater cost. Certainly, it may require a person 
who will deal more individually with the student's needs. Thus, 
the issues are:, a higher paid person to deal with "a smaller 
assortment^of specialized needs; and an individual person dealing 
with a smaller number of studejjis — an individual who might 
otherwise have b6en instructing a 'regular' class — and thereby 
negating additional staffing requirements. 

Many specialized classes exist based upon demonstrated need . 
(e.g. bilingual), demand (e.g. handicapped), and desire (availa- 
bility of assorted monies and grants). Oftentimes, the decree of 
support is based upon the degree of need; for example, an ^enroll- 
ment of 10 pupils may recieve gr^eater funding as compared to a. ■ 
situation with 5 students. WhiT.e funding may be contingent upon 
enrpllmen!|j; Expenditures often^re not. The difference between 
allotmer^ and need must be borne locally if the program is to , 
sfijrvive. Nowhere is this more demonstrative, than as realized ^ 
By' -the legisl^ated requirements of P.L. 91-9.41, and the reductioji 
in federal funds to help local districts satisfy these demands,*,. 

• . 'SI. 

/■ * ■ ■ -'r^ 

. ^ , • ■ 90 ■ ■ ■ 



Three alternative solutions to this situation would seem to 
bie: ■ . " . 

1. ^specialized teachers at particular locations 
with additional duties assigned to teachers; 
specialized teachers at particular locations 
• • with student collectiyely transported; and 
C3,; it^ teachers. . servi ng sever al locations. 

Unfortunately, negative measures (outcomes) associated with, 
under-utilization of. tdlent, rtcheduling constraints, .and wasted, 
transportation time ,dnd expense, militate against the final use 
of such remediatiort'strat'eqles,. 

Program consolidation at operating schools- ■•after selected 
site closures will servie to alleviate these dilemma. Less 
programmatic personnel will be required as well as a more efficient 
distribution of "utilization per. .overhead" affected. Specially 
trained personnel will be utilized%'re efficiently, transportation 
and time costs reduced, and extern4^:':fijnd1ng (as available) more 
efficiently allocated. 

General Program Staff Requirements and Re 1 ated Cost-Efficiency 

The final issue in exiunining fiscal^ expenditure requirements 
within the policy decision of school site consolidations, lies* in' 
the understanding of the distribution of salary and instructional 
costs across enrollment. Related tp the t3hys:ical plant and 
curriculum issuQS, general staff ing:feiinteres^^^^ nonetheless 
unique in that no inference is directed: towards qual ity (i.e. 
only quant1ty|is of concerrt^^S; , «^ , : 

Salaried^ teaching personnel gain no additipnaT remuneration 
if they instruct more students than another teacher within the 
same district; and teacher-management contractual gutdieTTnes are 



in force;, the element providecl the same 

salary whether fh^an under-uti.l 1zed or crowded school setting.- 
Very simply, with a ratio of 20:1 as^ specified by contract, four 
teachers with 15:1 classrooms could be replaced with three indi- 
viduals with the contracted 20:1 assignments. Thus, the union 
of ^class consolidation' with 'site cbftsolidation' provides a 
clear conceptual context for cost-benefit savings. 

: ' " 

t • .\l . . . ■ 

RegardTiess - of the Savings ... 

The above three issues of plant utilization, curriculum 
accountability, and staffing requirements support the potential 
of using school* closures as one alternative to a simultaneously 
declining'student population andjrfiscal revenue milieu. Taken 
singularyly or collectiv^y, it is emphasized.; that they are not 
the sole grounds for effectuating a consolidation strategy -- • 
only , that fiscal improvement and expenditure, relief are possible 
And the author concedes as equal ly important a criterion, the 
question: 

"How much improvement is worth closing a school?" 

As will be presented in a later chapter via an elaboration upon 
use of the decisioning^ model SCHCLO tthe SCHool CLOsure mode?)., 
a framework does exist for evaluating and contrasting criterion 
references in order to target potenti al. closure sites.. It is 
further maintained, that modeling neither emphasizes nor de- •. 
emphasizes any particular societal value or morality— in the 
t^echnical sense, that is. Rather, such often qualitative data 
must be weighed by the decision-maker, subsequent to receiving 
the modeled eva^ug^tion output and results. « - 
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CHAPTER 03 

vi 

E PLANNING STAGE 
[ Examining Basic Issues in Field Applications ] 
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The construction of a decisioning (slc)'m^del requires, that 

■ initially within the planning stage of such formulation a clear, 
understanding exist between t^e theoretical constructs of the 
model, and its design i^'satisfyihg these constructs. Or in the.; 
more proverbial ^sense, ^know where you are to -go, .and how you / 
intend .upon getting tKere— .before taking the> first step' . The 
planning for modeling .formulation is very much a stepwise, 

^ sequential ^process, not only in terms its actual, physical con- 
^truBl^ion, but also in light pf the numerous theoretical bases 
whith^underlie^^^^^^ tha^ construction. This chapter serves 

to highl,i^gj.ht these, preparatory guidelines; and to further serve 
as a 'field applications' primer for the novice — to illustrate 
the fundarfteiTtal linkages between the use of "multiple .alternatives 
•analysis* apd the 'multiple alternatives moder— and provide a 

.solid base for understanding the relationships between the 
theoretical cdnstru'^fs (i.e^ issues) on .the one hand, and their 
pragmatic, implementation (i. 'I: applications) on the other. 

Chapter three will .utilize the- "issue v. application:" paradigm 
as an fristructional technique in, present i ng • lihe nJefgor fs'sues which 

■ guide theCdesign and construct; on^pf tl^ie mult'ip1e.'*igilternatiyes 
model^.^and its ultimate utility in determining multiple a|terna-' 
tive* solutions. Four general topics wi 11 be addressed 'wfph in 

: this chapter: ' ■ " , • ^ 

. .... ;i\ :■ 



Ini1;ial' Constructs fiqr Under^^tandipa;:. • V 
::Inter-relationship of ;l^ul ti'ple-' Allferfratvyes 



:%it'^ - , • • ^ ....... ■ ■■■ • 

. ' ■ with Compet.ing C'riteriOn-Refereaces; ' " . 

, : - / Design Contingencies,, f-or Model -BuiTding; and 



ContrQl Contingencies for Model Execution. 

: These .^neral topical ^reas will. be discussed vi« nine specific- 
issues,, and the . illustrative field applications which demonstrate ■; 
'each issues' pragmati.c base-within educational evaluation and . 
■decision-making. ' ' i 

The first general topic concerning ' initial cor;istructsV wi 1 1 
examine the initial .foundation (and its critics,) for the modeling 
of policy alternatives, and the difference bet^en policy and 
actioji-alternatives ' model ing, the . second section concerning ' : 
'alternatives and criteria' will illustrate the use' of single.'' and 
multiple entry model ing^ and- the coniparative differences 'betv^en /' 
main and interactive effects formulations. 

Thd third part of this chapter attempts to define the' rel atl^^e^ 
Importance of two major constructs in model pTanning and ..design/: ^ 
that t^Jie model exhibit degrees of both mutual-excTusi venessl^nd : 
coTlective-exhaustiveness in its instruction. Finally, the area 
^of 'contVol contingencies' will be, highlighted via a discussion 
of measureable system impact, decisioning optimization, and the 
acceptance of preferences' and trade-offs.; 

• Wh^ile/chapter tw the use of multiple 

alternatives anal^sis^ within a specific; context for th? initial 
understanding of the r^er the' context of program budgeting 
ai|d "fiscal roll^-batrk? chapter three now commenc^e^s the more- 
tf|eore'tical. education of the. reader, arid presents, a discjjssion of 
n%& issues^^^^ applications) necessacy for a more . 

thorough. 'Understanding. . Chapters two arid three provide a prelude 
to the content of chapter, four which discusses the more technical 
and ^mechanical rudiments, of the multi pie . alternatives model ing 
framework. , : ^ 



' MOD EL f^lG TOLI CY^ rALTERNATI VES * / 



■ . AIT decision-making represen a- cons^cipus and demonstrative ■ 
• selection of 'cTie or more choices -of I' pr&f erred* action, f rorri ^a 
multiplicity of alternatives. alternatives 
vyhich jdefine ''the. policy /in^^app a specified problem area, * 

ior "action" alternatives; mettiods; of following the defined policy - 
and thereby implenienting its gui^delines. Policy modeling more- . 
...over,, r:e^Miires, /a ,strt^^^ part of- the modeler .that: 

first , nesp.bnsi b e' deci s^^ng^.^?!^^^ the ,^need to. project^ the^ 
futuref' impact of alterTia|i ve.^cfkibn^^ to their* selection • • 
and execution;' and s ebor^ ^^agcojjr^t able ^ e gf. these various 

alternatives demohstrate.^hat wh4ch. we •understand and -know of 
the effects, of today's de.g^s-idhs, .and tTieir utility for siffiulating 
tomprpof* ' s, decisional re.q^;i:emfents>. This is a true expositaon of. 
the * pragmatic research'' 'i^' Vutil i^zati'on-bas^d evaT'uati.pnV which 
is required -every emerg';i:rtg-e^^^ in today' s ever-changing 
worldj^ V ' . ^ 



•The Issue ^ 



The reali^stiq cynic invoTv^^.in/poTicy^evaluatio.n would - 
define, the ^nyironmeht idV |i8lfK|p orte •Wherein, 

."to>do one's able bestV with^ enabled to ^do." 

analyst would attest to the' as defining the^degree cif-' 

. ,^|,n.tell igence or^knpwledge the poi?icyvimplernentator br to the 
•i'f^eution of the.rble; and :td the' 'enabled^ a^ specifying, the '\ 
/(^c^iiStraints imp^^ by theoTenvironment upcyi the successful \. 
implemer^tation of the poTiciy guided . Indeed, poTicy ijs but a 
■ set of guidelines for 'assisting ^e administrator, in* eve^ntually 
•.obtaining :s^tisfactgry^completion of both persqhal and org^niza- . 
v^!,pnctT objectivesl; '-'^ . *. ^ * . ' • 




stratification of Operations Research (OR) Applicat 
Within Educationally-Related Environments. 



LIBRARY OP£RATIOMS 

TEACHER^SIGNMENT * 
PERSONNEL SELECTION 
GRADUATE STUDENT SELECTION 
STUDENT ASSIGNMENT \ 

FACILITIES LofeoN 

SCHOOL SITE PLANNING/COl^STRUCTION 

CAI COMPARISON .^??, 
EDUCATIONAL ADMINISTRATOR PF%\RATION 
COUNSELOR TRAINING % 
CURRICULUM REVISION ' '''.4 
TEACHER TRAINING " , 



VOCATIONAL EDUCATION/REHABILITATION 
HANDICAPPED/DISADVANTAGED 
WORK ENVIRONMENT/PRODUCTIVITY* - 
MANPOWER TRAINING 
■RELOCATING THE UNEMPLOYED 

STUDENT BUSING 

URBAN MASS TRANSPORTATION 

CORRECTIONS - 
INCARCERATION v. PROBATION 

ENVIRONMENTAL POLLUTION" ^ 
CHILD DAYCARE 

DEPLOYING EfCRGENCY SERVICES . ' 
STUDENT HEALTU SERVICES 
FOOD SERVICES * .. 

OUTDOOR RECREATION " ] 

DESEGREGATION 

AUDIO-Vl/uAL/MEDIA 

INSTRUCTIONAL EFFECTIVENESS 
ACCOUNTABILITY 

ORGANIZATIONAL COMMUNICATIONS STRUCTURE 

ENROLLM&NT PROJECTION ' 
CURRICULUM TIME BLOCK ASSIGNMENTS 

STUDENT MAJOR v. COURSE SELECTION 
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• Pol icy is . itself of course, an alternative- course of action 
which has been agreed upon by the leaders of .the -organization 
involved. It defines a set of strategies-. or oVer-ri ding. 'dogma 

'within whl^ch the organization must operate; and from which- more, • 
specific tactics or perf ormance -^specifi cations will evolve. The 
idea of "modeling'* is borrowed from trti? engineering scientrst, ' 
and represents a belief Ifhat careful, sfudy and' investigation of 
Vobserved' phenomena wi IT atde the careful deci si on-mak^i^; in 

• understanding potentia:l 'expected' phenomenon-impact basifid upon 
upcoming decisions. That is, the policy maker nee(Js' $t5m6Sy sense 
of what impact will accrue to the system as, -a whole, based upon 
the implementation of different decisions^ '"''The suffiqient outkw 
come of .'modeling policy alternatives' illustrates then for the'',\ 
decisioner, what diff erentra-1 benefit exists for the system as a. 
result of differing policies, and their varying effects; ^nd 
therefore matghes-for the system, the decisions of today with ^lie 
Uesired effects 'of the future.'* 

. (i ■ _ ■ ^ ■ ; . ■ 

The general idea of policy alternatives^mddeling however 
suggests 'at least three. different perspectives to be- addressed. 
First , such modeling can imply th.e need for modeling alter/iative 

■policies; that is, to study the philosophies associated with each 
of several alternative policies, and therefore to decide ulti- 
mately upon which philosophy will be defined as directing fiiture 
organizational efforts. Secondly , policy alternatives'' modeling' 
might .also suggest the highly structured modeling of the poter>^ 
tial 'action alternatives' illustrative of a decided-upon state- 
ment 'of polntiy or philosophy; that is, what actions are now 
possible based upqn the philosphy or policy adopted, and 
furthermore, how do these, actions now differentially provide the 
desired impact(s) to the'*system as a whole/ Finally , the 
modeling of policy alternate ve^s might well ^ddress the Vundamen- ' 
tal n^d for the policy analyst to. model the various 'contingency 

'^outcomes' Yor the potenti ar alternative actions now implement able 
based upon the philosop.hy or poitcy^ adopted." * 
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The^first idea .of m&deling parallels the input portion of * 
evaj'uation mo'deling: v/haf ingre*dients will be entered into t\\e 
organizational framework in/ order to produce th^ desired effect? 
The second idea best illustrates ^the process portion in evalua-"^ ' 
••tiye simulation: What procedures or particular decisions will^\ 
result from the stated phi losoptiV inj or^der to\satlsfy the iden- ^ 
tified needs and demands of the sy^em being mo^cieled? The third 
notion then spec'^fies the output portion of modeling: What 
results will be incurred bised upon the various specif ic actions ' 
taken as a result of the particular policy adopted? How the 
decision njodeler proceeds with the process o^ 'modeling policy 
alternatives', will largely defJend^ upon whicjh of the three levels 
Df ^specificity stated above, - t>€st illustrate the declared direc- 
tion of the investigation to follow. 

- The Application 

There once existed three rather large. and prosperous school' 
districts who began* to experience parallel problems of declining 
school enrollments 'and dwindling fiscal budgets. ' The first 
school district assembled their very d^\e policy analysts and" ^ 
evaluators, and decided that environmental impact statements were . 
needed to support the district's a priori decisions as to which 
school to .cTose — noting that this district had already decided . - 
upon ^school clqsure-as the policy^ decision to adopt. The criterion 
. of attendance zones' and its relation to student population centers 
represented the major (or sole) ^criterion for deciding which site 
was to be released from the roles. The second school district 
realized, that more than a single criterion would be necessary to - 
decide which school (or schools) should be closed,.* and so they 
also assembled a committee. However, tFris committee was comprised 
of community individuals from each of four sections of the district 
which mos^ likely would have to release a school ^ite, and consisted 



of such membership as businessmen, housewives and, various blue-collar 
work^s. To demonstrate true cormunity involvement, the 
superintendent charged this group with tv/o directions — the first, , 
to determine a set of criteria which woul.d then be. utilized to 
evaluate each of 'the school sites for potential closure — the second, * 
to submit a list of four' sites for potenti al.'release, based upon the 
results of this committee^basSd evaluation, 'r • 

Our, third school district initially studied the need for site 
closures as a means- of .alleviating the problems associated with 
dec.ltning enrollment and decreasing school revenue; aqd found this 
alternative to be the most' likely solution strategy ('.'policy alter- 
.na^Ne") to follow. Representatives of the district office, field 
admi/nistrators and classroom teachers, experienced university j / ^ 
educators in evaluation and decision-making, and community repre- 
sentatives were assembled to study the prob1em'."~~A "large~set" of - 

criteria (and their associated criterion-references) were drawn-up, 
and evaluated for their suitabi lity in determining ^school closures. 
Three models were specifically designed to evaluate the criteria, 
and their impact upon the deci^ion*(s) to be made — how many sites 
to release, anti which ones'. And finally, the models executed, the 
results evaluated post-hoc, and the results released for public 
scrutiny. > . : 

' ■ ^' ■ . ■ .V , " ' ' ^ V . ,^ 

Only one of our three .districts had a happy ending." Yet all 

closed schools as the policy response to declining enrollment and 
decreased revenue. What existence of differing modeling pr^'edures, 
(jr modeling outcomes, served to predict more success for on^ school 
district over the others? 

Although -one modeler was successful, all> faced the ^same ^problems/- 
associated with the fear, disb'elief and ignorance of th? general ^ -r ^ 
population. 'Nowhere-is this better and more clearly wi^nes^ed, -t^n / 
in the situation of 'program budgeting cutbacks^ as discussed within 



chap-ter two. The major problem 'to be ad'clressed' in all modeling ^ ^ 
episodes is th'at of criterion defv4niti on , refa^ncing and^ /iieasure > 



Criterion defitiition ihvolVes the illurninationv bf those ^reas" 
which- wi-11 serve* to compare the potential alternati ves for ^inclusion 
within the solution set.^ ^ In the problem of budgetary cuts/ criteria 
of need, demand, performanipe and pa^sonal satisfagtion are -of ten • 
defined for use in a cqmparaji^re evaluation. Criterion 'rej;erencing 
comes immediately 'hftgr definition, since once types of criteria 
are- agreed upon, 'sources of their measure must.be denoted in order ' 
to gain the evidence decreed by -the preliminary definition. Areas 
of Curriculum design .and -implementation will often have)Veferences > 
such as gain scores 'en standardized batteries to demJfe '(reference) 
^aj^rticular criterion ^ of performanqe. , Thus the results of each ^ 
student's performance on 'such a ^est will provide Input to the 
i na4-serl ec-t4on^of~Grfe-cur r4:eu-l-um-a-l-terna^^^^^ ei — ^ah oth er-;7-i n - th e - 
purs uit'of^ Instructional excel lence. '.>A1 though last "in the series, 
criterion measurement , is often the most volati le, and least defens- 
iBTe .for the mpdel builder. The measurement of performance via 
Standardized scores from an achievement * batteryos seldom- 
questioned, but not equally so for the needed measures of personal 
satisfaction. as might result from responses'to several Items on 
a 'disagreement - agreement' continuum. 

■ * 

individuals not understanding the model 's utility, or under- ^ 
standing it^well enough to rrot want its objective evaluation of 
the alternatives -r- will raise such issues, as true impact^not being 
.measureable, or that true impact can only*. b^ measured over several 
years of observation (rrot heretofore existent, of course).' These 
critics wit! declare that competition for scant resources Teaves 
the process open for hidden agenda andperpetrated bias, and^'that 
modelers will be able to use jargdn^d mum6le-jumbo to convince 
others oif. the validity of the mode Vers' decisilDns. The major 
argument— and the one most likeJy to destroy the utjlity of a 
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rdecisi'on model -for evaluating^potenti al deajl'lpGatioh alternatives 
for^ij^^lg^elf ar of the Organi:zation'.-- is one whicf^.states, that: 



.essent]aT to 



All program units now existent are 

the^missi^n of the^ organization; and therefore y ' 

Elimination or deallocation is an /unreasonable 
decisi onal alternative. Xhe more' prudent ^course 
of action is to reduce 'budgetary allowances by 
some percentage W^iijlt across-the-board, and 
di^rect each unit to, live-within-their-meafis. 



Such a^policy alternative^ls most likely/orfly^beca^use the necessity 
of such harjj decisions such as reduction-in-tprce of personnel will 
no longer be necessary ff%i the.upper-ec'helon of the management^r^ 



hierarchy. 



There is but .one pi^tection ag^iinst the 'arbitrary and capricious- 
judgements which 'the cynics and critics de/lare as dooms-day heralds; 
and that is the use of a clear^ alternatives-based, criterion- 
referenced^ decisioning model. Such a model must not only be able 
to evaluate b^th policy. and action altervative decisions, but also 
do 'SO in a complex environment which controls the worth jind desira- 
biH^ of each alternative, its' impact -upon the problem to be / 
solved, and its collective benel^it to the system as a.^hole. 



] 
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POLICY V. ACTION ALTERNATI 



One can view the polfcy. alternative^ to. be decisianed as the 



over-riding philosophy or direction' which an organization will 
adopt in determining futdfe; specific remediation strategies' Vor 
the identified problem s^iuatibn. Once determined; the specific ' 
action alternati v'es to implement the guidelines of the polity must 



"ITe^ evalu at ed "and" t hems e'lVed *deci s i oned . In^ effect, the action 
alternatives are the 'degree V of remediation* to be offerred to the 
•dire^Ction V of the policy being followed. ' 



The 'Issue 



— : — -Whgn--a~deci^-1xin-^ak«r^or-iD5m^^ 
/ courses bf^action» a basi ^ 



1^rf«t-4ve- 



istinciion must' be made between the 
' level .of specificity address cibr'le ^^the^ actions jjefined-. Recall 
tHat a policy repiresents a philosophy tp be, adopted and followed; 
whereas ^the' actions associated with particular policies represent 
. the implementation fostered by the policy adoptecj. Therefore, the 
likely, result 'Of pplicy-Tbvel modeling will be the adoption of a 
singular philosophy, within which the organization will operate.' 
At' another level, the philosophy adopted wi-ll guide the^organija- 
tion in successfully remediating a particular issue or problem 
facing the organization at that "time. ^ Obviously, 'different' philo- 
sophies may ^^^nstituted as dif^ent times,! depending upon the 
more higher-level or esgteric p^Tosophies .of the organization. 
For policy alternatives therefore, the result of modeling, or the 
simulVtlon of potenti al" outcomes, would most likely be the choice 
Qfi^-a_synqle policy or strategy; of actio n, , 




The second-level of policy aUernatlves* modeling however, 
aSdrgSses not the phi Josophy. decision per se.' , but rather the 
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likely actions which 'are alldwaBle based updn the specific policy 
or philosophy, adopted. Such action alternatives could be singular 
or' plural (i.e. multiple) in^scope; but address the implementation ' 
of the particular philosophy guiding the;organization*s efforts, 

-~Act4on- alternatives- are -f-ur^ther-d'i^^^^^ 1p their Implemen- 

tation for satisfying the defined policy; by whether or not ^the 
particular actions project a focused Attention upon, different 
aspects of the problem'being solved, or different ways, of solving 

"^e^^'en^^o'Tre^^^ rrr""enher case, the alternatives finally — 

selects^ for implementation may be singular oj^lural (i.^. one or 
more actions takeny. A further refinement of'th^plural c;ase,„lies 
fn deciding "Whether ^gach of the plural decisions require a single- 
entry or multiple-entry solution. - . . . 

The Application , 

^ • 

In the management of decl ine~enro1-linen1r,— and""its negative 
effect upon school district functioning, the wealth of policy 
alternatives could sustain a planning and evaluation staff for 

--nmaths* tW novice of t en~t)Trnk s of c1o sure„ v , no cTosure a s the 

two cilternati ve,s facing the superintendent. However, these are but 
the -tips of the iceberg, as it were. 

• \/ 

The complexity associated with the notion of closing a school, 
leads the decisioner to address the many subsets of such a global 
policy endorsement. If a site is to be released, will the school 
be mothballed, leased or^sold? If mothballed, how soon will the 
site be reopened based upM?a shift of the enrollment population? 
If the site is to be leased^;who will be prospective 'leasees? If 
the site is ,to sold, what wi.T.Tj.' be the conditions of sale? arid wi^l 



any controls be placed upon the type of buyer? 

Of course, the site might be retained for instruction, and 
other avenues of strategy explored for remediating the negative 
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•Impact due to declining student enrollme&ts. For example, a, varia- 
^ tion on the strategy utilized' during.>t:fvj enrol Iment boom days may 

be employed. That is, 'conduct class^es during a smaller portion of 
^h^_d^y (say, the morning h()urs), release the kiddies before lunch 

(/fR^eby eliminating associ/ated expenditures), and make the'school 
. , available to other organizations ciuring the afternoon!, during 

* normal business hours', and at some fee.. . ^ 



^And then there is the .situation where the decline of^student 
enrollments may only affect some selected grade-levels, isolated 
to a.particillar grading configuration (e.g. K-6, 7-9 and lQ-12).' 
Jhe reconfiguration of grade-locations (say to a K-5, 7-8 and 9-12) 
might also be a most viable alternative in solving several related 
problems (e.g. vacancies at one site^and over-crowding at another). 



But then of course, there is the usual policy alternative of 
doing nothing — "simple to follow, easy to design, .and best of all 
maintains the status quo. . V • 

. At the nex t. level from p olicy a lternatives, are the action 



' aTternatives -^to be evaluated to implement the philosophy selected. 
For site closures, the design of a model for determining action 
alternatives must of course compare each and every avai lable site 
for'' PQtential^osure — the major decisions being how many(?) and 
which ones (?). For grading reconfigurationy-each-antl-every possible 
combination of grading sequences must be explored, linked, and 
cVossed-examined. That is, considering the potential alternatives 



^Lj^K-ej-ij^^P;:!^ 

K-4, 5-8, 9-12' sequence, not only must- the individual segments be 
analyz9j^ (i.e. K-6 v. k-5T. K-4) for particular homogeneous 

-curriculum group4^^gsT-but--50-a-l-so-must—to^^^ 

addressed 'as a collective grouping 4K-4, 5-8, 9-12, as an example), 
in relation to feeder patterns, long-range forecasting and fu-ture 
conmunity planning (new housing starts, jnulti-f ami ly dwel 1 ings,e.tc. ) . 

, / ■\ . ' - . ^ ^ 

I - 
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Like alU sequent Tally-based processes, the choice of a policy 
alternative -must bd rigorous and final. Once- the pol4c-y. for- dealing-- 
with a problem is defined, the full energy of the organization must 
then be directed towards the^^ign and evaluation of retated^ action 
alternatives. As will be discussed in the. next chapter, policy 
and action alt^n^iti ves nnay both be evaluated as ""multiple" in* 

■ nature, and have the ability to be criterion-referenced and modeled. 
However, the decision-maker must never lose sight of the need for 

-S44mj4-^^ti4;g-^ ^ i m fiact-^f^ m e ith e r a p oli c y Q r-ac-t-ion^dec4-s4Qn— — 
and yn this way, be able to reyert posture and changje course If 
an unforeseen negative effect to the'sys.telTi becomes visible. 
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SINGLE- V.' MULTIPLE-ENTRY SOLUTIONS 



The conceptual difference betweej a closing or not closing a 
particular school, ^nd the notion of how many will be required for 
(flosing, and which ones illiistrates the cormnon' misunderstanding 
on the part of evaluators and decisioners regarding the potential 
of modeling in the multiple alternatives environment. No one or 
single decision exists in isolation from the futl spectrum af the^ 
organization* — nor from the impact related to other coterminous 
decisions ^l^ing made. Communication specialists in their organiza- * 
tional development models during the late 1960's, demanded renewed 
efforts for increased communication between and among'~tlTe~s^truc"tTire~ 

7 

of the organization. No less ^important is such communication to ; 
the design and execution of a criterion-referenced decisioning 
-mo^l .. wi^thi n mu l tipl e alt e rn a tiv e s an alysis. — — r — — 

The Issue * 

S 4-ng-T^-ent-ry-aet^^ns— suggest— a-^-singie^-respoh^ 
tory for addressing a particular issue or problem. This type of 
alternative is plural in the'sense that several 'singular' 50IU- 
tions may be required to provide a one-to-one response to the 
several problem areas related' to the strategic policy involved. 
What results in the final choice of a particular action then, is ; ^ 
one and only one actiorr<to be taken in the solution of each pSr- 

ticula r probl em-^-denti f i edl Combi n at i oriS^^Qf^ng 1 e-en try act i ons 

"or solutions wou^d exj^st.^^^ in as much as the several problems 
b^ing addressed were in some way related." 

^Multiple-entry actions or solutions in response to the imple- 
mentation off;:idQpted policy however, suggest" that a combination of * . 
„„ ^ct i^ons_ are requi red j n ori^e^ to juc^^^^ 
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area* involved. . Typically, a multiple-entry setting ajjdresses the 
area of decisibff mqtdeling, wher6in. some set of 'all possible 
decisions! must be analyzed; and ^ the -dual decisions of "how ipany-" o 
and "wKi eh ones" be finalized. The parti oil ar*dif"ficul ties asso- ' 

ciated with multiple-entry ".soTution fbmulati'Ort^^ Tie wi'tfiin the^ . 

dynamic 'relationship between these i interactive questions of size - 

and membership. /The size, of 'the finaj solu^tion set (that is, "the ' " 

numtjer of . alternative, a^tioWs required to meet pol icy. objectives) 

— ■ . ^ — ^ ^ , . ^ — ' — ■ ? ^ 

Obviously, depends upon the choice of actions to become members of 
'ithat same solution set. TWe chjoice of 'sorrte. individual so*iuti oils . ' 

may require a mucfr |sma;ller* set of /actions necessary 'for ' 
^ impTerfientation, 3s related to a ^smaller^set of othgr actions which 

wil l "do the-job" just as . satisf actor*i ly as^ the -larger-memtjership' 
\ set. ■ • , ■ ]■ . » ■ ; . 

Bw--sopM-s44^t-i-on-4n lin^-e r-^e n try sulubion ^ - 

determination is best understood in the required' 'dynamic* power of 



5. 



the sel ecti*^r';strategy.: STnce ln .effecfl, alT'ppssT&Te coWrnatTow 
■and permutatvons i^ potential solutions (actions) must be' analyzed, 
. a isituation of just three -potential alternatives might require. 
-sev^ri-^n<i1;ysesW-uur-dltei^^^ 



five potential solut'vons a total of 29 assessments. ' typi^afV cir- 
cumstances have been found to \asi ly incorporate 32 possible * 
, .multiple-e/itry memBer$h>ps (Whokben, 1980a) and 41 pfitenti al /mem- 
berships (Wholeben and Sullivan, 1981)4-., Indeed; it is well within 



the realn;! of reasonable imagination that situations of ^.several V 
hunjjred potential alternatives for multiple-entry cojnparisoras be \ 

jrequire d.; Such estimates increase exponen^tially as the de cisioner - 

_ considers the number of trite_riojijref_ere bei rig.. utilized ta^-ev^™ 
luate both the positive benefits as well as the negative by- 

prMucts .af„ 



The Apt)li cation 



"V 



loq_ 
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Since U is easily accepted in the modeling of schools for 
.potential closure, that the chgice of one school could Inadvertently 
'save' a neighboring school due to^the influx of student transfers 
from the closed^site — the realqj of a multiple alternatives, 
school closure model demonstrates the effect one member of 'the 
solution set can have upon- another potential member for^ that same 
set/ The issye of single v. multiple entry solutions is not re- 
served for ."the setting of policy v. action alternatives analysis 
alone. Indeed, a school closure model co'ul>d result' in the closure 
of one site only, and therefore be an example itself (aJthough not 
a good one) of a 'defected' single-entry modeling episode. 

A better and more util izable setting for single/multiple issue 
and its application* within decisional modeling, lies within the 
Vea of curriculum activity packaging, or general .goals/objectives 
packaging (Whaleben, 1980b). In a overly simplified example, 
consider the decision to be made between instructing either concept- 
A or concept B -- obviously, a single-entry decision; aad let us 
further assume no other concepts are to be comparatively evaluated.. 
Each concept of course, consists of a number of instructional 
objectives necessary to bring about the desired level of conceptual 
learning involved. But of course, each of the multitude of these 
goals and objectives have also associated with ,them a series of 
instructional, classroom (or homework) related activites which if 
executed, will bring about satisfaction of the goal or objective; 
and when all goals/objectives are satisfied, the singular concept^.K 
satisfied as well. 

Consider this problem: to determine whether concept A will be 
implemented to the exclusion of concept B, or vice-versa; and 
determining which objectives will be utilized to determine such a 
decision (where not all objectives will be allowed execution); and 
furthermore, which activities will be utilized to determine which 
objectives will be executed to determine which concept will be 
finally decisioned. Complex enough? 
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While the choiqe of concepts is obviously a single-entry 
matter, the potential grouping characteristics of the obj^tives 
at one level, and the activities at 'another level, illustrate the 
impact lower-ordered multiple-entr/ decisions will have upon a 
higher-order 'single-entry decision, /it is not at all trite to^ 
state* that one does not decide first the activities, then the 
objectives, and finally the concept — for such is the ingredients 
of an* irrelevant and therefore invalid decision, tn truth, the 
model must not evaluate the potential of the activities in. various 
groupings, but also jthe objectives in various higher-order groupings 
Thus, differential objective groupings will hold differential 
activity groupings, and all potential impacts of all potential 
combinations (or even permutations) must be evaluated for worth — 

exciting, is it not? As we will see in the next section regarding 

■ . « 

interactive effects, some 'more valued* activities may be discarded 
due to their inability to 'mix optimally' with other activities. 

For a more interesting and not-so-oversimplified example, the 
discerning reader is directed to the discussion of the decisional 
model for evaluating computer-assisted/computer-managed ^ 
instructional objectives across multiple micro-computer hardware 
and software packages — arriving at the best instructional p^ackage 
— in chapter eleven. . ' 
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MAIN- . INTERACTIVE-EFFECTS MODELING * ^ 

The control of multiple alternatives decision-making is^itself, 
a double-sighted context;. that is, choosing between multiply alter- 
natives, evaluated across^ competing criteria, and choostHg>Between 
individual and collett^'ve criterion impacts, resulting from the 
various multiple alternatives. Decisional modeling must preside 
for a strict visibility of criterion impact, both singularly as 
well as collectively. Without . such conscious control, an alterna- 
tive with high positive criterion measures qn a number of variables 
can arithmetically 'cancel' the negative effect of other -variables 
as the model evaluates the criterion measures. 



The Issue 

Modeling as a useful strategy for evaluating the potential 
benef4^"§ and/br shortcomings of a set of different solution alter- 
natives., must depend upon the various criteria which have been, 
selected and Identified to provide understandable^measures of sucij^ 
benefit or loss. While it is easy to understand the competitive- 
ness within the set of potenti al alternatives to seek membership 
within the final solution set (defining how many and which ones), 
it is also as apparent to perceive the parallel competing rela- 
tionships found within the set of criterion references utilized to 
measure each alternative's contribution to system impact, and tf^e 
various combinatorial permutates whi'ch result by varying solution 
set memberships. 

The most accepted form for displaying multiple alternative 
actions across competing criterion references is the decision 
matrix, where alternative solutions are Identified across the top 
of the matrix as columns, and criteria down the leftside of the 
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same matrix as rows. This simple*'2-dimensional design is suf- 
ficient for the single-entry case discussed in the previous 
se.ction. By imposing the. possibility of mull^i pie-entry solutions 

upon the matrix, and further statirw that the solution set WilT 

V 

comprise a total, of k-action alternatives, the matrix becomes a 
3-dimensional figure, with height defined-by membership. However 
to take. this analogy one step further, t|ie potential of varying 
membership size forces the addition of a fourth dimension, beyond 
the visual understanding of the average person. It^is in this con- 
text of 4-dimensional modeling, that the realm of 'multiple alter- 
natives analysis*, and its related dependence upon interactive- 
effects modeling exist. , * 

To understand interactive-effects, one has to first comprehend 
its counterpart, main-effectS;n[iodeling and the basal differences 
between the tvyo. First, it fs'^r import ant -to note, that main-effects 
modeling does not presuppose a single-entry decisioning context 
only. Main-effects modeling defines a procedure for evaluating 
potential solutions only in their contribution to a single com-< 
posite measure of system impact. Each potential action vvould form 
a linear combination of stated criterion impact (e.g. add 
'positive benefits' and then subtract the 'negative by-products'), 
and result in a single measure of potentialfor that particular 
action's influence in solving the problem at hand. Left relatively 
uncontrolled, are the effects to the system as a whole of the other 
potential alternative solutions . being considered. In the singular 
alternatives case, where one and only one solution alternative will 
be chosen, this is not considered to be a ma^r shortcoming. In 
the multiple alternatives case however, where the solution set mem- 
bership (size) is dynamically determined by the strengths and 
weaknesses associated with the particular combinations and per- 
mutations of the alternatives beiTi^ evaluated, main-effects 
modeling strategic^ are a serious compromise to solution validity 
and modeling i/eliab\il ity. 
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Interactive-effects modeling satisfies tlte^dual needs 
measuring (conti^Tling) both individuated collective contribu- 
tions of the multiple-entry solution to the defined systemic 
problem. Not only can. the variability introduced by varying solu- 
tion set memberships and sizes be analyzed and evaluated, but 
•moreover can the dynamic impac^ts of the identified postive and 
negative Influences to the system as a V/hoie be monitored and 
'controlled. These collective influences to the system, dependent 
upon the composition of the solution set, allow not only the poten 
tial alternative actions to be competitive, but also permit the ' 
various criterion references defining impact to be likewise corrr- 
petitive in assuring,, that the final sblution set membership con- 
glorms to the pre-established norms of the policy (philosophy) of 
the system — a previous modeling decision. ' . 

The Appl ication 

The process of selecting a micro-computer system for a school 
or district office is not'^s eaey as reading the brochures, and 
then buying what your neighboring school bought (anyway). The 
relevant potential of a computer system within the educational 
environment — for the full range of administrative, instructional 
and evaluative data processing options — is truly infinite. 

Consider a situation in which a number of instructional goals 
have been identified as being most amenable to presentation and 
drijl via a computerized system. Assume also that a number of 
administrative goals have also been delineated, in which many of 
the student record keeping, budgetary accounting, and inventory 
control functions of .the' main office could be facilitated by somet 
form of data processing system. Clearly, many usable systems do 
exist on the market ranging from the familiar TRS-80 and APPLE 
models to the less familiar OSI and HEATH sets. ' A variety of 
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options exist for eayh of the particular, models.^or systems; and 
a equally diverse number of integral peripherals are -likewise^ 
available to complement and exponentially ujDgrade any base-format, 
systems. Mhe quest i ons. become: how-to rel^ited the various 
instructional and curricular compon&nts to the various softwar,^, 
and then to the hardware system compatible >with thei desired and 
usable software; and , how to design an instructional system totally 
compatible wi^h ^e administrati vg requirements for a database 
manager. ' * * 

While chapter 11 deals with this issue in greater specificity, 
it is appropriate to examine some of the criterion references which 
might be utilized to perform some a high-level comparative analysis 
between administrative and instructional ^objectives, and their * 
appropriate satisfaction through the purchase of relatfed hardware 
and software^ 

Generally, it is obvious that some degree of measure will be 
required concerning such generic criterion references as need, 
demand, performance, eff ectiveness^^ efficiency and satisfaction ~ 
notwithstanding' the ultimate criteria, cost and avail abi lity. In 
addition, different views from diverse populations will be required 
in order to survey and understand opposing opinion structures. 
Therefore, it is likely that measures of opinion concerning the 
need, demand, effectiveness, satisfaction, etc. for various items 
will be forthcoming from administrators, teachers,' students, and 
quite possibly parents as well. In addition, it is further likely, 
that some items will be arithmetically differenced in order to not 
only measure the degree of opinion, but also the degree of consis- 
tency between potential user groups. Thus, teacher v. student, 
teacher v. administrator, student v. parent, and so forth must be 
criterion modeled in order to denote a measure of consistency^ or 
more api^ropriately, potential future conflict of interest and need. 
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The seasoned evaluator and decision-maker realizes that no one 
alternative mix between alternative objectifies,, hardware and soft- ^ 
ware-wilT be perfect.ly acceptable to all Jieeds ajid demands' involved. 
Therefore, a greater degree of closeness/ between the administrator 
parent .opinions may be seen as important as va high degree of con- 
sistency between teacher and student views. **Even if the teachers, 
and students do not rate a particular software package in science 
as the highest In quality— as .^ng as their. measures are at least 
acceptable, it may be better to choose the package on which their 
opinions were most alike. However, the combinations of objectives 
and software match-ups are numerous, and' different combinations or 
permutations will alter the finaV criterion measure of consistency. 
To choose the best match of objectives, hardware and software, the 
model must keep track of the level of consistency (in this^ example), 
the limit of consistency required and the minimum of singu^afr/ 
acceptability for each software item individually, and of course 
the membership of the particular grouping which best illustrates 
the highest degree of consistency possible. 

The* usefulness of such a model obviously, is in the promulgation 
of the above coritrol on' consistency, while also choosing the best, 
possible system within district (or building) expenditure limits. 

The modeling of interactive-effects is one of the paramount 
strengths of the multiple alternatives modeling technique. While 
it will control the selection of any particular alternative based / 
upon that alternative's criterion 'measures, the'^^nodel will further- 
more control for total impact upon the system, in terms of the fuTl - 
membership of the solution set decisioned. 
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. MU /UAL-EXdtuSIVENESS IN MODEL PLANNING ' . ; 

* ■ ■*■ • . ' . 

In model"" building,, altisrnati ves should be both separate and 
in'dependent in terms of their identification and definition — 
that is, be mutu41i^exclusive.of other alternatives. Without this 
contr€n' upon the' formulation of alternative courses of action, the 
comparative evaluat^ion of alternatives would be pitting some alter- 
natives against themse^v^s — somewhat useless" — but 'Certainly 
confounding to'^the measures of the various criteria involved. The 
pitfalls of non-mutually exclusive criteria — ^sometimes known as 
multi-col linear weigtrting — will also f)rovide a bias to the model 
which may!*J)?. uointended^-'Or' if intended, ^u^rofessional . ' 

The Is?ue ' . 

The modeling of policy alternatives, whether in the higher- 
order situation of determining general direction or philosophy or 
in the lower-^rder context ofv-determining specific actions in- 
satisfying the determined philosophies, requires attention be given 
to the degree of distinctiveness or independence between each of 
the alterna/^ives being evaluated,. . It is obvious;, that the modeling 
of very similar policies would be a waste of time —assuming of 
course, ^that this degree of similarity was acknowledged prior to 
the evaluation of alternatives. This question of similarity refers, 
to the concept of mutual-exclusivity regarding the construction of 
alternatives, and the related impact upon the decfsfoning model . 

The idea of economy in modeling alternatives is somewhat inno- 
cudus when discussing^ the similarity between alternative policies 
or implementation actions. However, the degree of collinearity 
between criterion references can potei^tially have a demonstrative 
effect upon the modeling process, and the solution resulting. For 
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example, if the' modeling framewock evaluating ihe alternatives does 
not control for the exist-ence of muTt^-co>linearity between cri- , 
teri^^used in assessing the valu§ of those various alternatives, * 
the potential for "stacking" the model i^ present thaf j|s. 



introducing relative idintical criterion-^ref erenced measures and* 

icuiar alternatives in 



thereby 'weight^.ng* the likelihood of par 
becoming members. of the final solution se 
come cpuld qbviously be by ^accicJent, or m 
design. 

The concept of mutual-exclusiveness i 
related to both alternatives construction 
utilization. ,t)verlap in the design of >,al 
unnecessary burden for the model during i 
mode, but is not necessarily compromising 
solgtion process. The .degree of similari 



^policies or actions is governed by the desire of the model builder 



Such weighting of 'out- 
3re impor*tantly^ by • 

n model planning is thus > 
and criterion reference 

:ernatiyes provides an * 

:s subsequent evaluation 
to the results of the 

:y between alternative 



various philosophical or 
degree of 'clear' ^ 
the mind of the model 



to design a 'clean* representation of the 
action-oriented alternatives; and by the 
understancjing of the problem at hand, >n p 
builder. . However, overlap with regard, to criteria inclusion in the 
assessment process can be a method for providing high probability 
of a particular alternative's final membership within the solution 
set. 



The Application 



The modeling of schools for potential closure is a fundamental 
example of evaluating mutual-exclusive alternatives. Although the 
fate of some sites may be inextricably linked- to the future of 
another site, this linkage is usually criterion related, and not 
a'%inctiqn of a 1:1 correspondence between alternatives. Exceptions 
to this rule may exist however, such as a vocational /technical site 
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related to a high school — but these are special cases easily^ 
.modeled, • 

Regarding alternatives and th"eir mutual-exclusivity, the major 
problem lies in curriculum evaluati^^^ wheje various instructional 
activities may possess. the idential criterion measures across all 
criteria (or at least, have been measured that way). Such 'toss- 
up' alternative solutions can be controlled via what we will come 
to understand as the "objective function" vector within the * 
multiple alternatives model. The multiple alternatives analys-is 
framework will accomodate such identicalcriterion vectors; but' 
the membership (if it exists) within the solution set can be only 
considered by chance\ The. true problem' of course is whether or not 
the two (or more) alternatives in question in. essence perform the 
identical same function, i.e. duplication. The model is not con- 
cerned with such potential of duplicity (although* it can be pro- 
grammed to do so), and could unwittingly select both alternatives 
as members of the solution set. Such a problem is ever existent 
within the assessment of curriculum activities and objectives; and 
must be guarded against for the sake of time efficienjcy and the 
duplication of expenditures. - ' . 

Regarding criterion measures, the best example of a conscious 
effort to reduce any form of duplication (or multi-collinearity) 
exists in the process lised to determine the final criterion set 
for determining school closures (Wholeben, 1980a). 

Using sources which ranged from business and industry, through 
past attempts at developing main effects models for site evaluation, 
to the. opinions and desires of key community and school personnel ~. 
a, set of 62 criterion measures were decreased to a final collection 
of 24 measures utilizing such statistical techniques as factor 
analysis and analysis of variance. Criteria were excluded from use 
within the final closure model if they duplicated an already included 
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measure from arrother 'supposedly unrelated* criterion, or if they 
tended to be segregated to a'particulv kind of site,' which was 
a priori determined not to be utilized as a criterion reference. 
By insuring mutual exclusivity within criterion references, not. 
^^or^y was potential duplication of criteria avoided but also was the 
use o£ unwanted biasing sources precluded. : : .._ 

^ For a more technical and manual-oriented presentation concerning 
the control of mutual-exclusivity in alternatives, the reader is 
directed not to skip reading the final section of chapter five, 
concerning the use of tautological inference vectors. Regarding 
the use of statistical procedures to help determine the degree of, 
mutual-exclusiveness in criterion measures, the reader is directed 
to study carefully the contents of chapter nine. 
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C0LLECTIVENESS-EXHAUSTIVENES5 IN MODEL PLANNING 
\ ' ■ , ■ . 

Controlled system totality and crtterio^representati vehess are 
the goals of coLlecti ve-exhausti vity regarding the prelimijnary 
planning for model Jdesign_ajid consJ:ructjo_n._ - sys- ' 

tem impact, all decisional[ units must be model regardless of^their 
potential, for inclusion within the final solution set. Similarly, 
the effect of a particular criterion reference can not be detected 
unless it was modeled within the original decisioning process. 

• , • • ■ a ■ 

.. ' . ' ^ ' 

The Issue ' ' : - 

' While the concept of mu^ual-exclusiveness focused mainly upon 
independence ^and overlap between both the alternatives under 
investigation, and among the various; criterion references utilized 
in the evaluation of those alternatives — the idea of; col lective- 
exhaiistiveness' is concerned primarily with the totality of the 
system b^ing- analyzed, and the universality of those impacts attri- 
butable to that! system. 

The col lective-exhausti vity based" upon the alternatives to be 
modeled, illustrates the need of a. modeling system to^ address two 
paramount concer*ns. .First, thfe .alternatives must in fact describe 
every^conceivable potential policy or action which has some degree 
of expectancy associated with the context being analyzed. It* 
should be noted, that satisfaction of this need for including every 
possible alternative, may lead to the violation of the mutual- 
exclusiveness or relatedness between two or more individual 
alternatives. This should not be considered a detriment to the 
model. A reliable rule-of-thumb is to exclude only those alter-- 
natives which do not demonstrate a different criterion measure on 
at least one criterion variable. Secondly,: this need to describe" 
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all possible solutions incorporates the notion that. Ss/s tern... i^paqt 
can not be 'accurately measured without including all potential 
, measures within the systemic modeling procedure. For example, a 
partix:ular^decision,ing alternative may not be a candidate for the 
solution se^tXdue to political, social, economic or religious, 

modeled in order to attribute.- its exclusion from the measure o1^ 

- total system impact. Likewise, a particular decisioning alter- 
native may be defined as necessary for inclusion regardless of the 

. _ ...measuces ..of -its -criteria- ---and- therefore,-- it^-T^^^ 

to the total system impact as measuijed by criteria, must be 
controlled. 

While col lective-exhaustiveness is concerned primarily with 
controlling for total solution likelihood and total system impact, 
respectively, in addressing the area of constructing alternatives 
for evaluation — the concern for the collecti ve-exhausti;i/eness in 
defining criterion references is just as important to the model 
planning process. With criteria, all possible references must be 
decided^ and included within' the model, to totally assess each 
- alternative's individual contribution to the system impact via its 
inclusion or exclusion within the solution set; and to assess the 
F^l^^^A'iys contribution r ^ multiple- 

alternatives and/or multiple-entrj^ solution, context. All possible 
criteria which are viewed to have a potential impact upon the final 
solution set generated must, be included within the modeli^i^g 
framework. Selective exclusion will have the same effect as, 

- included multiple measures of the same criterion measure for 
weighting purposes (as discussed above). 

' / ■■ ■: ' . ; - 

The Appl ication 

, One of the major errors in evaluating schools for closure 
.during th.e past few years, has been the conscious exclusion of 
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sites. which will hot be closed "no matter what". ;. Such^ a decision - 
is almost always pol itical , 'and sbrifietimes, even valid;. The problem 
stems from-the exclusion of the site*' from ^qqmpari.soil with other, 
buildings* susceptible to selection and thus; the' uncdnsciq^us' 
exclusion of the privileged site'$ criterion measures from being - 

-u-t4~14xed^-n-t4ie--f4fixk^-'5-ys-tem-w-i-deMm — 

situation, the final measure of . system impact in not, nepresentati ve 
of the total system; and is theref<ln*6 erroneous.. , Moael* can be 
designed to include a privi leged *site' s measures within the final, 
deci'sioning process; , and yet ^tautological ly excTlide that same site 
from the final solution-set {schools for closure). 

Col lective-exhaustiveness with respect. to criterion refe^ences 
howej^er, is a much more difficult issue to monitor and control* 
The experienced evaluatbr can well remember the* multitude of timqs 
the question' has been asked, "Have all the^neces'sary criterion 
references to make an informed decision been included?"; and the- 
equal 'number of times, crossed fingers behind back, the reply was, 
"All (I hope).#" Only time, comnunication and openness- can prbvide 
the necessary setting for di^scoyering .'all relevant* references. 

The necessity^f or full criterion representation is best illus- 
trated in the area of fiscal modeling for.determining program 
budgetary roll-backs. Defining the total system impact >.as those ^ 
composites which signify effects upon the var.ibus object . 
expenditure categories (salaries, benefits, equipments, capital 
outlay, etc.)^ the final solution of the budgetary- cutbacks can 
be modeled'^and/or monitored via examination of the final object, 
sums. For example, if the reduction of equipment expenditures^ . 
is considered to be of primary impQrtary:e, program with high need 
for "equipment purchases will be of weighted preference for inclu- 
sion within the final so'lutioh set. In the same vein, if a. goal 
exists to decrease salaries by' some x-percent, the final configura- 
tion of the solution set will allow a direct reference to that 
particular goal., " ' / 



Thus, the concept of coTlective-exhausti veness represents a 
defined need to not only model the full system' as it exists, but 
also, to assure all, criterion .measures desired/within the final 
decisional .process are present. Without such careful^ preparation,- 
measures of individual as well as collective system impact is 
impossible. " ■ . • ; 



INDIVIDUALIZED V, COLLECTIVE SYSTEM IMPACT 



Individualized system impact refers to those. criterion references 
which represent a particular' alternative's worth to the solution" 
set, and' to the system. Collective impact is a composite of .all 
solution set members, and nqnmembers — signifying the benefit of 
the arrlved-at solution to the system as a whole, and the remaining 
system functioning level. 

The Issue . ^ < 

As might be hypothesized, main-effects modeling within the 
singular-alternatives* context provides the simplext of all policy 
(or action) modeling situations. The concept of evaluating policy 
alternatives for variable utility is often conducted in a setting 
of multiple alternatives (or multiple-entry solutions)', and the , 
resulting reliance upon interactive-effects modeling strategies. 
Each alternative's individualized contribution to the system,^ as 
defined by the measure(s) associated with each criterion reference, 
can easily be recognized. Usually, each alternative will possess ; 
simultaneous measures of both positive benefit and negative ■ impact 
to the system as a whole, dependent upon which of the particular 
references are being evaluated. By establishing certain limits on . 
the •acceptability* of levels of such measurement, alternatives can 
often be excluded from further consideration as a potential solu- 
tion ~ that Is, having fai led' a particular criterion test. Policy 
decision-making In reality however, is hardly that absolute. 



hard enough, eich cind every potential alternative will have some 




Realism, and the belief in the fact that there are really no 
'perfect* solutions, will lead the (iecision-maker to understand the 
need to accept the bad along with the good. If one looks long and 
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criterion associated with it*s impact to the system, that will be 
'less than acceptable' -to the modeler and the system. Therefore, 
any modeling context must look to the 'big picture', and understand 
the total impact to the system as a whole — that" 1s, the-measure 
of collective system impact. 

No where' is this more important, than within the multiple- 
alternatives context. Understanding that no one solution will 
work, a collection of potential solutions is analyzed until the 
resulting 'set' defines an acceptable solution to the problem at 
hand. That' is, a group of alternatives will collectively provide 
the combined impact necessary in order to effect the solution 
required or desired. The modeling scheme utilized to guide th6 . 
development of a final solution set^must therefore provide for the 
control of such collective impact; and must moreover allow the 
necessary compilation of alternatives in seeking a collective 
effect. 

The Application 

Analogous to the creation of a subscale on a psychological 
battery via the incorporation of the responses to several items, 
the use of a multiple alternatives analysis framework as a ^survey 
questionnaire generator, has been studied. From a pool of^ several 
hundred potential items, various items (or questions) can be 
selected as part of a surv.ey questionnaire based upon certain pre- 
ordained needs or demands of the individual requiring the survey 
instrument. In this circumstance, viewing each item's contribution 
to a final "subscale" composite, and the total composite contribu- 
tion to the system as a whole — is directed comparable to any of ^ 
the models previously discussed within this chapter. 

^^''%1thin fhe itemization version of the multiple alternat'^ves 
modeling framework, criteirlon references are related to the degree 
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to which an item has been found to represent a particular view or 
response to a specific subject; and represents the item's contri- 
bution to a 'survey subscale'. Accordingly, the contribution of 
each of these *subscales* to the final instrument created by the 
grouping of these items represents the total instrument's contri- 
bution to the construct (or constructs) desired to be surveyed. 

All multiple alternatives models-- whether policy or action 
alternative oriented — possess the capability of controlling and/ 
or monitoring yDOth individualized and collective system impact, 
based upon the solution set constructed. In the school closure 
model, this is related to assuring that the schools remaining open 
contain enough seating capacity to accomodate current enrollment 
levels. In the budgetary roll-back model, program budgets are de- 
allocated in order to meet the requirements of a specific, dollar 
reduction in revenue. In the selection of a micro-computerized 
system for both instructional and administrative objectives, it is 
the stipulation that one and only one System can be purchased — 
and that this system must be the best in, terms of maximally satis- 
fying the prestated objectives. 

It remains these ideas of 'best' and 'maximal' which. a decision 
model must somehow operational ize in its systematic pursuit of a 
successful remediation strategy. 
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OPTIMIZATION OF SYSTEM IMPACT 

To make the best decision possible without creating new problems 
or more realistically without creating new problems more difficult 
than the initial ones being solved, is the goal of the decisional 
scientist. Oftentimes, the best decision does not- seem to bring 
relief to th6 system, only to redistribute the negative impacts 
-somewhat. The model can not solve the problem; it "can only in 
essence display the relative merits of a solution it derives from 
evaluating the alternative options input to the model, based upon 
the criterion measures describing those options. 




The Issue 

Accepting the negative aspects of a solution along with its 
postiye effects, is a key concept within the decisioning modeling 
of policy alternatives; and is referred to as systemic optimality. 
Basically, the model .is designed to adopt a specific solution set 
membership, based upon the criterion measures defining the range, of 
alternatives included within the model, in such a manner that, 
positive impact to the system as a whole 1js maximized , and negative 
hy-products based upon that same solution minimized . The extent to 
which the system will be allowed to assume negative impact is based 
upon the model's imposed limits for collective negative effect to' 
the system as a whole (discussed within the previous section). 
Once again, the modeler must reljf upon the conceptual framework of 
the decision matrix. 

;/ With a matrix specifying individual alternative's performance 
'across all criterion references, any combination of alternative 
solutions may be collectively viewed as to their collective impact 
to the system as a whole. The initial modeling context is able to 
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further specify the level of positive impact which must be attained 
or surpassed (i.e. maximization); and denote the level of negative 
effect which will be tolerated as a limit (i.e. minimizatefon) . 
Such dual demands of maximizing positive impact and minimizing 
negative effect is known as defining thfe feasibility region within 
which the potential decision(s) reside; and therefore constrain the 
model's search for a 'best of air solution to the defined problem. 

The best solution (multiple-entry or otherwise) will be one 
which initially does not violate any of the constraints placed upon 
the feasibility regi'on; and which subsequently provides the highest 
positive (or lowest negative) impact to the systan, as measured by 
some one or more optimizing criterioji references. Tlris is called 
cyclvc; optimization, or the cycling of various demands through the 
model, and the' r&lated observation of how such differing demands 
influence the variable development of different policy 
alternatives. The final, or optimal solution thus portrays a reme- 
diation strategy which not only assures strict adherence to intro-' 
duced criterion measures and their limits, but a'^o provides the 
best mix of alternatives which . collectively meet the overall 
demands; of the system modeled. • ' 

The Application 

.• 

A solution is optimal insofar as its represents the best-mix 
of criterion measures utili'zed in describing that same solutfon. 
The multiple alternatives setting is the clearest example of the 
constructs underlying solution optimality and integrity. 

The 24 criterion measures which were utilized to determine the 
solution set for site closures, controlled not only individualized 
impact to the solution but also collective effect to the system as 
a whole. Of these measures, 13 were designed to demonstrate a 
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positive, desireable Impact to the system — and thus were 
maximized (e.g. area of the school, current enrollment, history of 
survival percentages, proximity of enrol lees to the site, number of 
• minorities, avai lable classrooms, and- physical characteristics in 
ratio to utility consumption and expenditures). The remaining 16 
measures were construed to be negative in scope, and were therefore 
minimized (e.g. student residence overlap between attendance areas, 
site overlap between neighboring schools, site ag^). 

Using these criterion measures as defined above, the school 
closure model was designed so as tojjj^monstrate a solution set 
^ which would impact the system (distjpict) as described; that is, 
close schools with smaller areas, history of survival non- 
retention, and so forth and* thereby keep schools with larger 
capacity for, enrol Iments shifts, closer student re^jidence proximity, 
and so forth. With this model, 24 individual solution sets were 
designed .in such a fashion, that given the model as controlled by 
the 24 references input, each criterion reference in turn was given 
status as guiding the deivelopment of the solution — that is, given 
the weight of maximizing or minimizing various positive or negative 
impacts, respectively. In this way, each separate criterion was 
afforded 'fair advantage* in guiding the model to selecting the 
best-mix solution of .schools to close (both number and name); and 
each solution set was compared for consistency across all iterati'l^ns 
of the modeling execution. 

To optimize all criterion measures in sl^ a way as to construct , 
the best, possible solution set, is the ultimate goal of the 
multiple alternatives model. ^ However, it again must be recognized, 
that the best solution may^not represent all of the individual 
criterion impacts desired by the initial modeling design. 
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PREFERENCE AND TRADE-OFF CONTINGENCIES 



Since decision-making is often a form of col lective bargaining 
and personal arbitration, it is relevant for a decisioning model to 
contain features of such selective compromise.. All decisioning 
contains criterion-related ingredients of preference ~ that is, 
for the final solution set to manifest particular, desireable 
qualities. Equally relevant then, is the need to incorporate the 
ability to 'trade-off* certain virtues in order to arrive at a 
solution for benefiting the system as a whole. 



The Issue 

■ In planning and designing a modeling strategy for evalua"|;\ng 
policy alternatives, the model building must be willing to accept 
Mess than normally* acceptable* solution alternatives based upon 
measures associated to one or more criterion references. 
Furthermore, any reasonable model must incorporate a mechanism by 
which the decisioning .process ( viz ., the building of the solution 
set) is able to analyze and determine preferences based upon the 
interaction of the criteria with. the combination of potential 
/alternative actions. 

Preferences can be modeled into such' a framework via the use of 
weighted criteria; that is, allowing greater. influence for a select 
number of criterion references as opposed to the remaining measures 
utilized. In the same way, trade-offs are modeled through use of 
collective measures' to signify system impact as a whole (i.e. mazi- 
mizino'-'thevpositi ve, and minimizing the negative). Since it is 
likeny that *perfect.* solutions will be impossible, the model must 
be aMe to understand the limits within which it will^operat^, and 
the su&sequent listing of both positive as wel.l as negative impact 
to the sysWi, based upon the solution formulated. 

* ^ 



The Application 



This final issue in chapter three provides constant fuel for 
the critics of quantitative,, decision modeling. The fact that 
many final, solution sets will not .exhibit all of the modeled cri- 
terion constraints upon the solution, leads the novice to believe 
that the framework has failed, and that its use is therefore un- 
warranted. ' . 

All decisioning has preferences, as illustrated by the direction 
given to the various criterion measures — to maximize the positive 
impact to the system,, and to minimize the negative. The development 
of a solution set, will of'ter incorporate some undesireable measures 
in order to allow the positive impact which would result. This 1s 
the acknowledged existence of trade-offs as represented within the 
results of the multiple alternatives model; and It is maintained as 
a strength. of the model to be able to make such' traderoffs as it 
progresses toward building a solution set. 

In a post-hoc evaluation of, the school closure model for 
example, it was^found that all criterion measures which were to be 
maximized, were — «and likewise for those to be minimized. That is, 
the soUation set displayed a consistent array of 'negatively' 
oriented measures as opposed to those schools which Vemained. open. 
However, some closed sites resembled many of the opened sites on 
one or two criteria. These sites were decisioned as closed based 
upon the predominance of their impact as measured by the criteria, 
and therefore upon their impact to the system as a whole. 
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L Multiple Alternatives Analysis as 
a Mathematial Oecisioning Model ] 
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THE TECHNIQUE 

: ■ ■ • ■ ; ■ I 

'To include or not to include a more detailed and technical 
decision of the mathematical design within the MAM — was a long 
•and arduous decision for the author. On the one hand, inclusion 
of a technical section (I reasoned) might sensitize the general 
reader negatively; and preclude that reader's pursuit of the 
remaining text. On the other hand, exclusion of that same sec- 
tion (I rationalized) might very wel V undermind the final accep-f' 
tance of this report as a valid technique. '! 

.The fin^l decision to include at least some minimal amount of- 
technical development was made, based upon four premises: -j 

(1) though a technical decision may in fact threaten some 
readers,, the:^ultiple Alternatives Model Is- (also rin 
fact) a technical design, for which I neither minimize 
nor apologize; 

(2) a technical discussion will add credence 'to the ' 
operating mechanisms of the model, illustV^ate its inte^. 
working p,arts, -and promote a detailed unders^i^nding of. 
the 'Vinput-process output" relationship ~ far above any 
"Trust me, it rea^lly.'^works!" manuever; 

(3) . the technical formi^lation can bje both,.1nformatiye ::ahd 

documentary, without re^ading like a ■bibchemis.t's; report 
on the postpltuitary hormone, oxytocin (that is, 

* ^43'^66'^12^12^2v interested); and 

(4) a responsible reading; . of .CKapters two and' three has 
already acquainted the reader; to the general ideas con- 
tained within Chapter '4; that in fact, Chapter four , 

A ■ ,/ 
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should be a •recall* episode for the material already ' 
alluded to in the previous f isc;at1 discussion. 

The presentation in Chapter four consists, of three sections 
which follow a very simple introduction, design, and evaluation/ 
validation paradigm. The discussion is void of any particular 
problematic context; and thus is more generic.than system- . 
specific in scope. Such- an inductive approach, viz . from the " 
specific in Chapters two and three, to the, univers^ in^Chapter 
four^ will hopefully not only promote better learning ^and . 
understanding of the MAM^J^cllnic^4eyT-^^ the 
T^der to review the manuscript more closely for its intuitive 
and generaltzable applications to other problem areas of' . 
education. ' ^' 



INTRODUCTION. TO THE. MULTIPLE ALTERNATIVES MODEL. 



The Multiple Alternatives Modeling (MAM) framework makes four 
assumptions' of the problem (decisioning) areas to which it 
(model) is to^e applied. First , the pgroblem is a-multiple 
alternatives p'roblem, requiring a; multiple alternatives solutidn. 
That is, the solution to the specified problem situation could 
reasohaly call for the itijplementation of more thari 'one of, the. 
alternative courses of action being evaluated. Whether. -it be . 
schools to be closed, budgets to, be cut, programs to be ; initiate, 
or routes over which to transport'students —greater than onei . 
scho.Ql',f'43udget, program or route^'may ba selected as the solution^ 



Secondly , crJteriort" reference pointy (j/^. , variables) c,an be 
quantitatively measured for each of the defined alternatives, 
demonstrating an alterative's impact (if implemented) to the 
system, according to the criterion's derived focus. Furthermore, 
this arithmetic summation of all 'selected' alternative's.^ty^.i- 
ter-ion :yalues (across a partifcular criterion) forms a c^mpOrSt.te 
numerical value which illustrattes the sol ution'^s impact (selected 

/ . • j S 

alternatives) to the system as :a whole. 

Thirdly , the system being modeled can afix some high (or |ow) 
limits to these criterion..§uwnatfons, caTled'upper^ (or lower), 
bounds. ^ If a criterion meas'ures the totaU.cost of each prbgr^^aip 
.being considerM/or implementation, and a total available btid^^^^ 
of some specified amount exists — then the summed total of ^^^^^^^^^^^ 
.program budgets (for the program to be implemented) must be equa¥ 
.to.or.Hess than the, total available budget. ; Obvious lyj^ you^^^ 
not sp^rid more than\ypjj have aval lable (although program agmij 



. nistrators do it rel^J-grously) . In this example, the I^^T-bud' 
available .is seen as- an 'upper bound'. Similarly,a 'lo!^|^ 
could be- the total amount to be cut from an operatAiig^uiSei^t-, 
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where the crlteriop -is the cos't-i:b*be;--saved fbr each of the 
potential alternative programs (budgets) available 'for roll-backv 

Finally , some one. Individual criterion measure Is identified 
whitfh will be utilized to optimize-the selection, of /the final/- 
alternative mix-set (solution. Many seti /of alternative 
is, combinationa,l permutations) can ' Usually tie Jdeittified w^^^^ 
will provide a solution to some degree or exteritv, /However, 
Reality -normally requires aivfdher.ence to some priorities 
^ v^'^^^^^"^ within the system being modeled; for example, a desire ' 
^' •^ to nraximize the number /of students transported (on the average) 
,:..:;'v.i.a each bus;, or a des^ire to minimize the number of stops,^ a bus • 
•>y; ttas to' make enroute tp/th'e/ school . . / .> - '■ 

The Multiple Arternatives Model. >V a complex response to- a 
•/'"complex decisioning situation. The-'mqdel recognizes the need to 
simultaneously evaluate all available; al/tex^natives across al l - 
defined criteria, and to therefore simuTate^^^^^^^^^ interactive • 
. nature of a criterion-inferenced, . dec/i/st^ environment. 
Above all else; '.the MAM framework prpvtdes^^.j^^^^^ fo^r eva- 

luating a set of alternatives, col lect.ivel.y; -^^^^^^^^^^ based upon the 
set pf criteria which^^the real-life .■d'eci.Vion-maker^ have^p 
as the desired ingredients of their final decision; y 



Role of Multiple Alternatives in Decision-Making 



In the multiple^lternative/s context, the potential solution 
vialternatives may be displayed as a serial string;'^ /^^^^^^^^^ 

H-^Z ^2 "' "l^^""^ '^j represents the' jth alterati.ii^e^" (of n 
tpt;;al)>, j.= 1, 2, n. The MAM decision is/to i.nclude (or 
exclude) each alternative ■al' a member of the fiha,! solution ' 
(mix-set). The only value which xj may assume, is (that -is, 
to... include ) a '.0'/ (that is, to exclude). Therefore, the decision 
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is to matfiematically assign either the value of 1 or* 0 to each of 
: -^^^^^ j = 1, 2, ...» n; thus the label,, binary ,.. . 

/inta^er progranrn.'^ng,. 

In .each case -of ten a^rlterriatives, the serial representation • 
wpuld.be illustrated, as: [ ; 

. C ^1 ^2 ^3 ^10 ^ 

.: If the'flna,! solution inqluded alternatives 2, 5, 9 and 10 as 
memb.e^^^ the solution vector would be displayed as 

^ ■ ' CO' 1 0 .1 0 0. 0' 1 13. 

' As we will see in the next section, the function of a binary 
' coding (0,1) extends'' beyond its'use as an easy display mechanism 
for. aHernative solutibn mempership, . . . 



Decision Criteri.a as Moc|etling...Co'nstraints 



We have sp^t) that an alternative becomes part'W the solution 
by taking on the value^6f«-l (that is, Xj = 1 for some j-of n); as 
opposed to the value d|.C;i(xj = 0, for all j of n, such that Xj ^ 
1, j = 1,' 2, n). THe basis >for..i&^igning I's v. 0*s, lies 

in. the evaluation of the criteria which were selected and 
. ';;measOred to indicate each x.-'s impact upon the system being 

modeled. Furthermore, it was the summation of the crjterion^A^ ? 
valuesojacross ;.1:he selected (solution) alternatives which f 0f - ^ |; 



mulatecj^the mult.iple alternatives solution (mix-set) impact to 
the system. * - 



Let us define an 'a' as Vepresentirig'vthe valae of any cri- 
terion for any ' alternative. It is relatively straightforward 



t'tKep^^ -'Interface a ^ . as the' value 6f the ith criterion's measure 
f6r/:the..j.t^^^^ ;.Fpr. example, recall our previous 

example; oJ^. ten alternatives. If there exji ted -only one criterion 
to assist ;us in evaluating the. s^et of potential- solutions, then 
the criterion values could.be represented as: 

■ ; ■ ^^1,1 ^1,2 %3 ^1,4 ;4V../I,ip'.? 

In a more complicated example, a set of three criteria used to - 
evaluate ten al$#tfat4'Ves 'Would be represented as: 

^1,4 ••• ^I'.IO^ 
^2,4 ••• 32,10 ^ 
^3,4 ^3,10 3 

The first case involving only a single 'criterion, is called a 
vector of criterion valueis?; across the potential alternatives. 
The second case where three criteria existed, is called a matrix 
(i.e., a collection of two or more vectors) of crl'tcrion values 
across potential alternatives. Since most MAM problems involve 
more than a single criterion, and because each criterions' 
measure'' will be utilized to constrain the decision to be made (or 
more appropriately guide the selection of alternatives for incj^iir 
sifln within the solution set), the matrix is known as the -v: 
constraint matrix . 

At this point, w"e have introduced the .vari ^bles of Xj(.j = 
1,2,..., n) ani^ -1,2, ...,m;j - 1,2, ...,n) to represent 

the alternatives and criterion values, respectively;. that is: 
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t;fi,i 31,2 ^1,3 

^' ^2,1 ^2,2 32,3' 
f ^3,1 33^2 33,3 
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I ^^11: ^X2 X3 



Ut criterion: ^ j ; V ^a^^^ • -ai,2 ^1,3 /'.:. , . a^,, 

2n4 criterion; j a2^i a2,2 a2,3 ;:a2,n 

• , I . • 

I . - 

nith^cnterlon , | ... . 



Can you .s§e (?) that a further refinement of the above scheme 
could be made to appear as follows: 

^ ^1,1 xi 31^2 ^2' ^ ai,3 X3 ... ai^n 3' 

t ^2,1 xi 32,2 ^2 32,3 X3 32, n 3 



C am, 1X1. an,,2 X2 an,,3 X3 a^^^p Xp 3 

This makes sense if you recall that: (1) the value of each 
Xj will be either a '0* or a '1*, depending upon whether it is 
excluded or included within the solution set; and that (2) the 
sum of the criterion values measures the total impact of the 
solution upon the system. ^ . . 

Consider a relatively small example of four^ alternatives . 
being [T}6asured across three criteria. Thus the modgl would be 
represented as: V , - 
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^1.1^1 + 31,2^2 + ai,3^3 + 31^4X4 

VP'lj'' ^2,2^2 * ^2^.3X3 + 82^4X4 

I ' ■ 

^3.1^1^+33,2^2 +33,3^3 + 33^4X4 



(system impact if.CRIT #1) 
(system impact if CRIT #2) 
(system impact if CRIT #3) 



Now if the solution vectpr [x^ X2 X3 X43 was represented as 
[1 0 0 13 where only alternatives #1 and #4 were selected, the 
model would be shown as: 

ai,l(l) + ai^2(0)/ ^1^3(0) + ai^4(l) 
^2,l(l) + ^2^2(0) +32^3(0)+ 32,4(1). 

^3.1(1) + 33^2(0) + 33,3(0) + 33^4(1); 
or. in reduced form: , n 

^1,1°'^ 31^4 (solution impact if criterion #1) 
^2,1 32,4 (solution impact if criterion #2) 
^3,1 * 33^4 (solxition impact if criterion #3) 

We now see why previous discussions of tradeoff /preference 
and interactive-effects were germane to the MAM development. 
Note that if fl 0 0 I3 is to be our .soVution, then the values 
of .raij,..a2j, a3j3 exist for x^ and Cai^4, a2^4, a3^4a- for 
X4 = 1. Thus the solution Xj, j" = 1,4 requires that we accept 
the criterion values of"(a^,^j, a2^j, a3j), j = 1,4 ^hether they 
are most desireable or not. What we also know is that the choice 
of Xj^ and X4 as solutions must coincide with the upper/lower 
value restriction placed upon the criteria. 

i 
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System Demand and System Impact 



The limits placed upon each of the composite measures fbrmed 
by summing, each criterion's impact across all solution alter- 
natives (i.e., Xj = 1 for some j of n), reflect two closely 
related, system-stimulated components: demand- and impact. 
System demand exemplifies the need(s) of the system by the 
demand(s) placed upon the value composites of each criterion 
sunmation; that is, the upper or lower bounds of the criterion 
sum across the selected solution alternatives. However, since 
the bound is established only as a limit (not to be violated), 
then it is reasonable to assume these sums will seldom be equiva- 
lent to the bounded value; that is, the composite may be somewhat 
less than the established upper bound, or somewhat greater than 
the established lower bound. The actual value and its distance 
from the bounded value, is the measure of system impact (f or eac^ 
criterion of the alternatives selected as solutions). _____ 

Based upon the already linear relationship between:, the cri- 
terion values and alteratives (defined as coeff i cieritS igd inde- 
pendent variables , respectively), it is a simple -ex^^^ to 
model these criterion limits as a function of inequalities. Thus 
in a threealternati ves, two-criterion model {requiring a 2x3 
constraint matrix, right?) — where the first jjriterion has an 
upper-bound required, and the' second criterion a lower-bound— 
the representation may be stated'as: 

^1,1X1 + ai^2^2 31,3^3 1 '^1 

" ^2,in ^ ^2,1^2 ^ ^2,3^3 1 '^2 - 

^where b^(i =: 1,2) are the upper and lower limits of the first and 
• second criteria, respectively. These values b^ are known as the 
values' ^of the right-hand side (RHS) of the constraint matrix'; the 
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values in vector format (bi.bg) are referred to as the entries 
of the conditional vector . Therefore. if the solution vector ^ 
[1 1 03 is to be analyzed, the following algebraic relationship 
must be satisfied: 

• .'jj . . . 

• '^1.1^1 +. 31.2^2 14 ■ 

If moreoyer a particular, constraint (criterion relationship) 
exists such that an equality is required, the linear equality: 

is useful.' arid ^val id. a . 



ine. ptvnty of linear equalities and inequalities in formulating 
1:.he^^^^^ model is. obvious. However, it is reasonable 

'^a/^P;;^^^^^^^ which more than one set of alternatives pro- 

^■,■:;*ytdes^a -sdlijt^^^ to the MAM problem. In these. cases, additional 
^; system-^pritDrit^ be set. 

System Priorities and the Objective Function 

Consider the circumstance where in evaluating the above three- 

alternativ^e, two-criterion problem, two plausible solution sets became 

evident: il 0 I3 and [0 1 I3. Sitice both are plausible, we now 
that each of the relationships: 

^1,1^1 + 3^3 1^1 • ' , , 

^2.1x1 + 32.3^3 1 ^2 
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and 



2,3x3 + a2^3X3 > 

are individually, simultaneously satisfied. The question 
becomes: how to choose between the first ([1 0 I3) and second 
(£0 1 13) sets? 

The MAM framework provides a solution to. this dilemma, via * 
the use of another crier ion called the objecti ve f uncti on . .( or ^ ^; 
cost vector ) . Unlike the criterion constraint inequa,11;tieSp;::t^^ 
objective function; does not have an est abolished upper;\(or^^ 
bound assigned.. Rather the* criterion coefficients >fdr' the .ob.tec^ 
tive function (labeled Cj, j = l,2,.,,,n) are summed and the 
additional deman established^ tfjiat either a maximum or minimum sum 
be found. Consider the following scheme: " 
■ * . . . . 

(Alternatives) 



^2 



RHS 



criterion #1 
criterion #2 



^1,1 ^1,2 ' ^1,3 



^2,1 ^2,2 ^2,3 



(objective function) 



C2 



^3 




If the criterion referencing the objective function was of a ''^^y\/::.\ 
positive-consequent nature, that is mea\uring good effects of 
each of the potential, alteratni ves, ^then it is reasonable to a.- 
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desire a maximum value from: the summation of cj -bashed upon the 
alternatives selected as solution. ^ 

/ 

/ 

If the solution [1 0. I3 is selected, then the evaluation 

■of: ' ...•"^ • • 

........ . , ^ ^ 

C^Xi + C2X2 + C3X3 

results in the composite: 

Likewise, the. solution set [0 1 1] results in the composite: 



C3 



we 



If are positive (i .e. , good and desireable) measures and 
therefore wish to maximize the summation — it is intuitive 
that the greater of the ({c^ +-03) and + C3)) values will 
decide the final choice/between the [1 . 0 I3 and £0 1 I3 sets 
respectively. That is, if in fact (Cg + C2) > (c^ + C3), then 
the relationship;:;. 



maximi ^e ,c,, x-i'^'Jc:^^. +,,CoX 



'n^^f^p. >;[r3,^3 



yields the solution set cQ^sl^.i^vW^h a;||?^'p||i!d: objective 
function of (^c>, ,,The ide^G# ||MjS|ni^ h^ "good") and 

minimizing (the ''t$l9%:1t;^le summaii^^ function 
coefficients, .'detfionstrates the issbfi^^p^vigi^ feasible 9 

solutions. Both [1 0 .li -^d; .[0 - 1 |l|ij^g|^- 1^eas i b 1 e solutions 
in that both satisfied the limits established via the inequali- 
ties and the values of the RHS or conditional vector. However-, 
the 1^ 1 .1] solution was optimal as. it alone maximized the 
objective function summation. 
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In summary, this example could have been stated completely in 

the MAM framework as follows: ■ , . 

■ ■ * 

to maximize : Cj^x^ + ,C2X2 + C3X3 

^2,1^1 * ^2,2^2 + 32,3^3 I ^2 
' ■ , . • = (0,1), j =1,2,3. 

The next section will focus in greater detail on the actual 
quantification of the coefficients; and the development of bounds 
for the conditional vector. 
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; ""^'^ure 7, -^f [esentation of the Augmented Decision Matrix .Model 
• as .he i^lultiple Alternatives Model" (MAM). 
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Constraint: =04 



Constraint #05 
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(Cost Vector Coefficients) 
(ir .-'ILP, X is inleger; if decisonai, ;c=0,l only.) 
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DESIGN OF THE . ^MULT-I-PLE ALTERNATIVE MODEL 

Sfmply ^t^ted, th^^^ Alternatives "Model is a cbllec^^^ 

tion of ^simultaneous linear equations and inequalities, wjtj^^^^^^^^ 
additional string of serial values (the obj^ective f unct1t)h: ) 
available tp "break any ties" wMch result when mover thaia orieX 
vector of solution values (0,1) exist sVor the independent., • 
variables (the multiple alternatives). General ly speaking,' these 
equations and inequalities which make up the constraint matrix 
and conditional' vector (righthand-side) could be further labelled 
' as the dependent- variables (Jhe foci of the particul ar criterion 
constraints ) . 

~ ihe coerticients of the criterion constrainLs, Uie . values, 
reflect measures for each of the xj alternatives (j = l,...in) 
across each of the defined 1-criteria (i = l,...,n). The 
b^ values (i = l,...,n) of the RHS(right-hand-siae) represent 
the limits (upper and lower bounds) placed upon the sums of each 
criterion, summated across all selected (i.e. solution) 
alternatives.. Since a selection means that the specific xj value' 
will equal 'T, then the criterion value a^.xj (or a-j- times 1) 
forces a-j. to be an addedito the'sum.. 

We may now'improve tremendously up6n our earlier charac- * 
terization of the decision matrix. By adding the ideas of the 
conditional "vector (to insure flexibility and the potential for 
tradeoff /preference), we are able to model the interactive- 
effects premise required of multiple-alternatives decisionlng. ^ 
Supplementing the model further with an objective function, the 
' , set of feasible solution alternatives can be further analyzed to 
choose the singularly best alterative mix- set— the . optimal • 
solution. Figure''7 displays the scheme of the augmented model, 
within the original eight-al ternati ve/f iveconstraint milieu. We 
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may now proceed to discuss in greater'-cJetail the nrciasurernent and 
operational schemes which are possiblie (and desireable) within 
the MAM framework. . ' /. 

. • >•'■'''■'> V 

Generation iSf-^M 1 1 e c t i v e 1 y- Ex h au s t i ve A 1 1 er n at i ve s' ■ ' ■ ' 

Valid construction and.i^^^^^^^^^^^ execution of the MAM system 
requires the user to recall certain rudimentary facets of basic 
research and^ experimental design, and statistical analysis the 
issues of c'til'lecti ve-exhaustiveness and . m.ut'ual-exclusivehess . . We: 
will delay the,- discussion of the mutual' exclusivity of seletrtedv- 
alternative s&lutions until a 'later^a^tion. ''.However, the issue 
of col lectively^exhaustive altern atives is germane novy. i ■ v : - 

' Recall the two 'strong iiuits'* (obvijDusly among many) of the 
MAM framework: (1) control of" interact! ve'^effects modeling, and 
(2) measure of total system impact. Interactive-eff ect^modeling 
provided a control over,, the rather complex' mi lieu produced, when 
decision-maik'ing was evaluating multiple alternative solutions 
across multiple competing drft^^^ Single alternatives could be 
measured as "good" or "notrsd-good" on varK)us criteria - coro- ' 
pared to the correlated crlteripn.^ measures ^ other alternatives,^-, 
and analyzed as to which alterait.ive displayed the best-mix of ''-.-^^ 
criterion measures. When compared, however, did some set of 
alternative solutions produce a better "interactive-mix" than 
another set? And, were different amounts of alternatives 
required for either? Thus, interact-|ve-effects modeling assumes 
highly complex interrelationships '^ngfi^^only between sets of 
multiple alterantives and among (within) those same, sets; but 
also presumes related interactions between the criterion measures 
of. .ttf^^ same alternatives (both singular and multiple) --the 
necessary foundation for a trade-off /preference potential. 
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The.otherv^^ of MAM .qenters about the issue of 

Ae^suiHig^' impact- . You -will recall, that the. , • 
/RH^rValues/p^^^^ upon the;!Summation of each <;rj"t6rt^^^^ 

■.■Gonstraint. --thus limj^ the^imfj^act: M the selected- ^Tter- . 
nat1ves,;WiTT^be^^"^^^ to fbstfer^. '"However, RHS-Values 

establish either an upper-or lower-bound to the summation, not ^ 
tfhg actual value which the summation must assume. Therefore, the 
trjjie sum for any criterion constraint .might very. will (and 
usually j^^^^^^^^^^^ b^) different. i:han the' pronounced limit; that ijs, 
somewhat ; less >than the up^er^-bound,. or somewhat grealf^r than the 
lower bound;^ 'In this way, the desired impact to the system is 
cpntrolable but mdreov^^^ actual irfipact 'to the system, ts ' 
iTieasureable. • By knbwing^the discrepancy -between the desired and 
aictjU.al -impact, the MAM njodel can be used, tp' detect changes to the 
"sy ^•te¥'^i .^TV^dif f er e h t i' al ; viinp ac tj^wlri^cjrrti^ ur^hroug h^he - 
: seljsit!^ different aTterniatiVe v > . 

It' should be. obvious to the reader, that' rte^^'th^^^^^ 
of interactive-effects nor the- recogn^ of system impact, via 
varyi ng alteranti ves * conf i gur ation^,. ' ii; poss^^^^^ without the . ; • 
existence of all possible, feasible, ahd'relgveht .alternat 
courses of, action for consideration. More succinctly, the set of. 
multiplgtv^aiitera^^ evaluated across tKe de^^^^^^ criteria 

constraints rrius't exnibit. the characteristlts of 'a 'c'ol lectfvely- 
exhausti ve.'popul atiorf of alternatives. The exclusion of any . 
altenl|itive forp the model, automatically precludes its impact 
upon the evatuation of the remaining alternatives, and its impact 
upon the system as a wholel ) 




Criterion Measurement and Constreiint Formation ? 

' S>:nce the criterion bonHraiht represents a linear relation^ 
(either equality or inequality) = of ^ 

or more concisely: 




— ^i j (<. =. >) bi for each i =1, ... m. 



extreme caution must bft used in .developing the a^j coefficients 

9^ .variables. '^Obviously, each a:jj must be , >; 

.numeric; and f urther ^xht^ as to anowTtheir 

arithmetic sum to'be a rational and. useable quantity for com-' 
parison wit,h the associated b^) RHS-value (discus^^^ 
sectionlv : \,. " 

Four basic scaling schemes exist f oromQasuring and encoding 
data: homi nal , ordinal i interval and ratibv : ;Pro^^^^ 
inclusive, all can be utilized to formulate Vtlie. a^j coefficient, 
dependent upon the def iriition of the "palrticular criterion ,%j 
GonStraj^it (i.e. its focus) . The ikos^ common scales ujttl ized' are 
"the interval and ratio measures, due to theij ability to compute 
measures of central tendency (arithmetic means) -and distributive^ 
variation (standard' deviation)* We will 'iimit our discussion to 
these scaling techniques on.ly. 

Data concerning program expenditures (e.g. in-'abllariunits), 
number of required p^rs5nnel (ei'g. in TTE-units), or energy con- 
sumption '(e.,g. 'in BTU-uriits) 'are easily fatlOrScaled measures. 
•Other data whi ch jni ght be , obtained from samp>6 ORi nibnnalres con-- 
cerning'ihe respondent's perceptions toW^'ds each particular 



alternattyes might easily be interval-scaled (6.g.'a six-point 
continuum measure associated with'a 'Strongly Disagree, 
Strongly Agree' resfeh^e format). * ' , / /. 

Consider the objective: ... 

"To deallocate sucji ^t)nogram alternatives as will secure -an 
• expenditure savings at least some'"*amount '$SAVE'^," 



Clearly if we-.cost-out each, program alternative and arrange . 
the constraint asrfoJlows: . . . 4 gi 



then the solution vector [x, Xg, X3 ... x^3 must be of such ;(0,1) 
configuration as to allow the sum of the $. ^ t.a b^ at Teast the 
amount $SAVE or greater. This is orie of the easier examples of 
the use for a rati.o-scaled.'ci^'tterion.^^^ 

Consider artb,t her objective fdr use of the interval-scale: 

"To deallocate such program alternative^l^ as^^wi II coincide ' 
with the public's opinion of • each program' s relative *lack of ' 
merits" * . . . . ' . 

/ . ■ ' - • / ' . ■ 

Suppose that a questionnaire was sent to a random sample of 
individuals, wherein the question. 'was asked: 



"Program x^ fulfills the needs of 'the community- 
to which it applies."' 



.and the response tallied via the use of, the following format: 
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strongly Disagree. .Moderately Modei^tel'y, Agfee •Strpnglj' 
/Disagr*£e . * . Disagrjee\> ' Agree "^^ ' Agree ^ 

where.ci lessor magtiitucie (ij^e^.l, 2., ) dlsplayed^the 
respondents dUagreemenJ^ with the itertf el iciltor, and thif> 



measured a negative* respons^g to tjie percevved met^it ^for each iof 
the [Jrografhs. ,If 100 pelfple responded tg»* these iterus tone for 
..eibh p»ogram alternative), the 100 percept iprT5-(l, 2, 6'"^^ 
> could be averaged jnd compiled into the constraint serial:* 



PCP^., xi. + PCPi,ix2,+ .... + PCPi,n >^n' 



where PCP^ \s tjie ith constraint, and represents the .group's 
(N=100) perception (PCP) of worth or mervt for each ^prograrfi. Of 
' course, .a value for -the RHS (b. must be computed; and we will i 
survey this development in the next section. ^ , 

The importance ^of .each criterion constraint within the 
constraint matrix'lies in its 'ability to.model each alternative 
singularly, (via the individual a^^j values), and collectively , ; 

(via the summation of the a^j, j=i ry)-^ Singular modeling ( ' 

allows the individual alternative's >cont.ribution to interaction- 
,,effe;cts to be input to the decisioning model. Collective mode- ^ 
ling then al lows the 'impact to the total system of potential 
solutions to be comparatively evaluated against the established, 
bounds of the conditional vector. ^ 

^(/omputation of the Conditipnai :(RHS) Vector 

n . • ■ 

*. * . . 

• ■ s - ' * * 

The-need to limit tKe constraint- coefficient summations, for i 
t;ealistic simulation of the system being modeled, as wdll as the 
use of ffhese same summations to detect system" impact, should now 



be (hopefully) obvious to the reader. If one wishes to limit the 
extent of some negative effect to bhe system via the alternatives 
selected, then some upper bound is establishedfor-the.sum of 
criterion coefficients which represent this .hegati^ve impact. 
Similarly, if the modeler wishes to force some lavel of positive 
impact, a lower bound would be defined for the suiri of * positive* 
:criterion coefficients, stating that this minimal sum must at 
leqist be attained. What may not be so clear yet, is, how these 
limits are arrived at. • 



O/ie of - the authors has , performed considerable research in the 
different ways to develop the RHS-values in the conditional 
vector. These efforts have produced two basic methods for 
generating the RHS (the first, static , meaning to be established 
a priori, and therefore non-varying; the second, dynamic , meaning 
to be defined algebraically within the model, the value(s) • 
varying as the model varies in its search for a solution). The 
most conmon method is the static approach because qf its ease in 
modeling arid the acceptability of its- assumptions.' We will limit 
oiir discussi on at this time therefore, tdvthe static RHS-value ' 
generation technique.^ 

First let us review what we" are attempting to ^ accomplish with 
the linear inequality. Coefficients have been asisgned to each 
of the alternatives-solution independent variables (the multiple 
alternative decisions being analyzed); based upon the focus and 
intent of the. particular criterion being modeled (as a 
constraint). * Ecexution af the models will sum various subsets of 
the set of coefficients defining that criji.erion; and will 
repeatedly compare that sum to the RHS limit assigned, in the con- 
ditional vectors; that is: ,i 



^^-^ a^.j xj £ b^.^ ( negative impact) 
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on. 



a-j^- Xj >; ( positive impact). 



Recall also that the value is to denqte, impact to the system 
as a whole; that is, the collective, impact of the subset of deci- 
sion alternatives being evaluated as a potential solution. If we 
think of "as a whole" arid "collective impact" in the arithmetic 
sense, a useful analogy is the arithmetic average or mean. That 
is, the system will average the coefficients' value being 
analyzed; and -co^are this average value with the sum of the 
coefficients. ^ 

To accomplish this, the modeler first needs to predict how 
many decision alternatives (x^^) are^^ikely to be selected for the 
.final solution; let . us say 'k/\ Then the modeler computes the 
mean of the coefficients for a particular criterion. 



and multiplies the mean value A. by the expected number (amount) 
of multiple solutions, 

. ^ y 

where b,. is the RHS-value to be compared with the constra'int coef- 
ficient (subset) sum. Thus, ^ ^ 
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where the number of j's approximate the value of k. 

Unfortunately, experience shows the use of the mean (alone) 
to be substanti al in establishing a workable b. value, this 
problem is easily rectified by introducing the standard deviation 
of the criterion coefficients to the b. formula. Remembering 
that the standard deviation SD^ is obtained from: 
' • . ' \ . •<•>■:■' • 

i I _(Ai-.a,j)2| 

we can now modify the b. generation formula as follows: 
UPPER BOUND b. = k • (A^. + SDa) 
such that, 

n " 

aijXj < k • (A^ + SD/^) . 

• j=l 

LOWER BOUND ,b. = k (A^- . SD;^) 
such that, 

n 

tise of the mean and standard deviation provides a consistent for- 
mat for c(Jnstraining the declsioning matrix; and each constraint, 

<j ■ 
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therein. Not only is systematic flexibility afforded to the 
model as, it searches for a unique (optimal) subset of solution 
alternatives, but the problems associated with initial system 
infeasibility are minimized. The notion of infeasibi lity will be 
covered in greater detail in a later section of this paper. 

Cyclical Optimatization Via Iterative Objective Functions 

In our earlier discussion of the rote of the objective; func 
tion (or cost vector) in gaining the most optimal solutlbri^-^ 
(decisipning alternatives subset) from the available- feasible ' . ■ 
solution subsets (acceptable to established constraintsKrthe'v 
reader may have /become aware of the subtle bias the "i. 
OF-coeff icients place upon the final solution. Often, this bias 
is intended-. As often, however, it provides "fuel" for model 
critics to attack the MAM procedure as another "computerized, 
mathematically gerrymandering" technique. A satisfactory solu- 
tion to this 'potenti ar problem is available, and relatively 
easy to implement. 

The idea of cyclical ORtimizatjon involves the cycling of 
each constraint entry (i.e. -the coefficients a^ • of the 
constraint matrix) through the objective function; Generally 
speaking, this involves re-executing the model i-times, once for 
each Qf the defined constraints where: 



C. . = a. . 



each i-th iteration. 



. Si'nce we either maximize or minimize the sume. of the 
Cj values, depending upon their positive or negative focus 
respectively, cyclical optimization must be structured to then 
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maximize the objective function when the constraint values being 
cycled aare of a ^positive impact* nature. And of course, the 
reverse being true for**the negative-impact constraint fo^s. 

The MAM structure thus involves an optimization stra^te^, 
which may be depicted as: 



n 

MAX 

MIN — 



CjjXj for each ith iteration 



n. 

subject to : a^-jx-j (<, =, >) 

j=l 



^ij " ^ij ^ of jn; ^ 

Cyclical optimization .does not el imiriate the b.i Js x)f- 

tive function; but rather allows each constraint^;f opus; to simi^;''/- 

larly bias the result of the solution sub'set'se'lect'ion/ Asj/W^^^^^^^ 

will see in a later section on ''solution .t^each^ng;" the cypVfn^ 

t)f each constrair^t thrpugh the objective furtjcttdn^ prqV:^^^ 

useful technique for studying total system jmp^ct.' - '^^ 

Evaluation of'the Multiple Alternatives Kodel = I- -'^^^-I'^^^y-: 

After rather detailed treatments * of . the .design a^^^^ constftic^^^^^^^^^ 
tion of the MAM framework, additidnal" discussion cdncerriihgi-'.-t^^^^^^^ 
model's evaluation (that-is, implementation; ■and.:e>iec)Lit^ 
seem redundant to the reader. Obviously, the -model *.is^ 
in full acknowledgement of the way in which .TtvwTl.l'^ 'woi^k:^^ 
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select alternative solutions. And the authors have gone to con- 
siderable length tp indicate how the model will react to the " 
vaHous changes in its clesign and development. 

But execution of the Multiple Alternatives Model is nbt in 
itself a "static" process. As a system of simultaneous linear 
inequalities, varying configurations of the solution vector 
[xj X2 X3 ••• Xpl will produce different interaction effects 
among the criterion constraint coefficients, effecting directly 
their sums and thus their^.ultimate comparison to the ^established 
RHS-bounds. It is conceivable (and unfortunately occurs often), 
that the initial relationship, between the constraint matrix and 
conditional vector produces what is known as an infeasible 
region. The model must then be revised by relaxing one or more 
of the constraint s^ in order to determine^ 

(locate) initial feasible space , and its associated parameters 
(b^ values):; / Since relationship between ;c:r1t^ 
a1 ternat-i ves ( v1 ^ . ; -/inter acti ve effects modeling) is not Irmne- 
•diately evident in an infeasible situation, considerable time can 
often be expended in locating the '^problem" RHS-value (or 
■Ivatuesyw, 

vP:l,-Execul^1o^^^ refers to a previously discussed "hotidrn of 
Tcy^^^ differing solution vector subsets 

:j;Wh'f^;ii-^^i^ function is replaced by • 

v'd^fife^^^ must keep track of the different 

r^tu^^-^^e the term, solution teaching )/ and observe 

^th^:*!^^ impat upon the system's final 

c\SOlutTQaV^:rS-M^>i'^^?^\^%^:-- ■ ' 

^■'■.-'•^ ' iv;-:-i--'^ "^'ir- . - ■ - - • - - 

^^^:."ilf^^^ still sound like "Cicero's ora- 

- tibp: not errored by including 

•//th^.svJsectixiifc^ • i: ■ 



Total System Impact Via Multiple Competing Constraints 

As each potential solution alternative competes with other 
alternatives for inclusion within a solution set, so also does' 
any particular permutated solution subset compete with other 
feasible solution vector alternatives. The formation of the =v 

optimal solution vector.-.occurs as the system asks itself these 

■■■■■'•'v .■' ■ ' '".^ ■ ^ 

'.cjuestions during execution: i^^^: '-^'^ 

(1) how many alternatives will occupy the saTQtion vector? , 

(2) which alternatives will be selected? 

(3) will these (e.g.) three alternatives better:;!fit the 
constrained- system optimal ly,-.vversus these other five? 

: (4) will in fact any combinational permutatfbn of the alter- 

natives being modeled satisfy the constraints? 

■ • \ - .; ■ fl ' ' ■ 

(5) which constraints "constrain*' ■rnp^^^^^ others? which 

^less•?■■f^" 

(6} if ;t^ ve^ctorVis comprised of 'Idesjired 

impact, J how close can the model select an optimal solu- 
tion vector, and minize the desired v. actual . 

n • 



(b^ - : a| jXj) values? 



j=l 



(7) what tr|iiiepff s/preferences have been made based upon the 

criteria as a whole, in the selection of one feasible 

solutiq^ subs'et over another. ' . 

Thus, impact to the system being modeled is bsed upon both 
the selection of the solution vector subset and the related cri- 
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terion constraints, ^It is this complicated interrelationship 

between alternatives and cri teri a;( viz > , interactive effects 

mo/Jeling) which makes the MAM an outstanding criterion- • 

■■ ' . . ^ 
referenced, decision-making tool. 

Initial System Feasibility and Constraint Relaxation 

The MAM framework cannot systematically evaluate solution 
subsets for an optimal configuration until system feasibility has 
been initially establ ished. Feasibility simply means, that at ' 
least one soTiition vector configuration exists W.iijch will satisfy 
the modeled criterion constraints (linear inequalities). If no 
'.such configuration exists (that is^'the splutiohfvector a zero 
^vector ib 0 0 ... 01^ :then; the. system^ infeasible. 
v.^:; : subtle, the occurrence rpf a zero-vector is a moSt 'impor- 

;tant (albeit, frustrating^ ^re^ult^ 

If the system^ has been carefully ^I'mul ated and modeled via 
■valid criterion constraints, with the RHS-values accurately / 
reflecting system needs- and/ or demands —then the result of a . ^ 
zero-evctor simply means that no alternative is acceptable to the 
system as a solution. In most cases, the modeler would thien ^ 
"relax"* one or more of the modeled constraints by increasing an 
upper-bound and/or decreasing a lower-bound . Such alteration 
makes the selectitjn of some solution vector'easier without 

violating a constraint coeff icient summa'tion.^ 

system modeled (in reality) can neither reasonably nor ration all y 
accept the relaxation of its "standards and priorities," the 
modeled region is declared infeasible. The modeler must then 
seek new potential solution alternatives to be included. _ia the_-_ 
MAM framework. But if the earlier issue of the collective- 
exhausti veness of the alternatives has been addressed, the system 
is declared insoluable. ; - ^ 



Cyclical Optimization" and Solution Tracking 



I , . Cyclical optimization is accomplished by utilizing the . 
a-ij coefficients of each- constraint- as the c-jj coefficients of 
the objective function. For a model with m-constraints, a. maxi- 
mum of m-executions, each with m-different sets of objective 
function coefficients, is possible. During any particular opti- 
mization cycle, the '^constraint whose coefficients form the objec- 
tive function-is still retained as^'a constraint for the ;• 
de termination '. O^^ initiar system feasibiyl ity; ^ ' 



' Quitev/obtibu^^^ (we hope) thisi- reader now'ponders ^ fact, , . 
that m-executions '{tiK^^^m^^^^ m-optimizationsj will 

produce mrselis of optimal solution alternatives . That is, \iveh 
a fi ve-aVternativ6 model, with three-constraints, the resulting 
three cyclical optimizations could result in the solution.^sets.: 



X5 - , TOTAL 







X2 




H 


Cycle 1: 


0 


0 , 


1 


0 


Cycle 2: 


: 1 


.0 


0 ' 


1 


Cycle 3: 


0 


1 


0 


0 


TOTAL 


1 


1 


r 


1 



I 

1 I 2 

'0 . I - 2 

0 1; • 1 

. I 



A s yo u can--see^-tbe-^;^^^st--two^c^ set-wtth^ 
two 'solutions' each; the third Selecting a single alternative 

only. What is more interesting* (though suicide-pro-vok-iag_iji^ 

real-life) is the fact, that eaeh-dlternati-ve^was~ctTuos-^^ 
(and only once) throughout the three cycles executed. Which 
alternative(s) should then constitute the. solution set? 
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TOTAL: 3 1 ^ 2 ' 3 5 ' 

' ■ • ■ " ' ^ .• 

This five-alternative, 5-constraint problem has produced a series 
or '3-3-3-2-3' sol litions throughout the five .optimization cycles. 
More important (fori now, anyway) is the total frequency with 
■ which each alternati/e was selected as a member of the solution 
* set. This franiewoiiy is very close to a vote casting situation, ' 
where each of five voters can vote for a maximum of three 
candidates. (We guess voter #4 is as frustrated as many of us ' 
are at election time;) Candidate #5 "pi cked-up" a total of five 
votes, foil owd by candidates 1 and 4, Candidate^ 3 and 2 
acquired tow and one vote(sj, respectively. ' ^ 



Such a tallying of solution choice by constraint cycJLe__i5_— 
known as solution tracking . Each set^of constraint values takes 
a turn at influencing the development of a solution set; at the 
end of which, a simple tally displays the proportion of total 
choice across all possible alternative choices. . The novice at 
this point may declare that option 5 is a clear choice; and that 
l,and 4. should follow suit, 'forming the solution set: 



ERIC 



Depending "ijponJ'the mutual", exclusivity of :s6 alterriatiye' 
solutions, we chose to,' agree amicably^or disagree violently. 
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Mutual Exclusivity, of Selected Alternative SoTutions . , > 

We have devoted an earl ier .siscti on tO: the" import anee Qt for- 
mulating collectively-exhaustive alternative^' ) ; 
"eval uati(in~v1'a"the MAm framework. It -was- al i& expressed^1:ha^t the, 
al ternatiVe? should overl ap as li ttle ^s pqs$1 b.Te ( if at all ) ; 



that is, the alternatives should 'represent. 

actions — no portion " of "'which are included withj^^^^^^^^ 

ahot her - al tern at i ve so 1 u ti on being - evaju ated i Such" ! di st inct i. on 

is Icnowh as the mutual excl ustvi ty .of cdef Ined al ternati ves , 



It; remains ironic then, tovnow state that '^some multiple 
alternatives problems by their .very naturis and substance, 
priscrude such mutually-exclusive solutions, ^At this point^of • . 
ambiguity, the best teacher is an: example..' ^' ' . . 

Consider our earlier cyfclicaT optimization /illustra^^^^^ 
which the final composite solution vector " was: V .' v 'y a ^ 

' " C^l ^2 X3, X4 'X5I = t3 1 . ? 

-N^W IP t n r. [ U^i^-th4J;-Jva li i tinn 1ntn /rnntftxt, As^ a /pol icy al te^ 



native to the management of enrollment* decline," you have chose to 
evaluate five elementary school, sites foreclosure, based upon . 
five a 'priori stated criteria. Being the intelligent and far-, 
sighted persQn you are (you're welcome!,), you decide to invoke 
the MAM framework to analyze these sites. The resulting five- 
cycle optimization produced the above composite vector. What is 
your decisipn? , . v 
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Those of. us experienced in, school closures (you can tell by 
"the scans) know, that 'the closing^of one site may" preclude a ' 
neighboring '' jeopardized' site from .imnie.diate closure, due to the 
transfer of students form the former to .the latter°.,school . Thus 
the choice of one alternative (e.g. site) may preclude' the '. 
rattonal selection of another alterntiave. This more si tiuat'ion- 
fpecific i^Justrition of a generic nonmutuaT-exc^sivenesv. - 
inherent in the problem itself, demonstrates the rieed for the 
. mocJfiTer to beware. , -r: • ° ' " 

''A solution to this problem is evident, however, ' Choosing the 
5th .site for closujre,^ the RHS-value's . . ■ - 



can be, recomputed with i = 1,...,4 (5 missing) and the problem - . 
re-executed. Of , cqSfirse, . the enrollments of the school's -neigh- " „. 
boring the 'closed site would be increased via transferred ' 
students; and criteria ..wher^.enro factor, recomputed. . 

Also, site #5 would no Monger be a part of the rtiodel. ' ^ °' 

. ■ - .■ ;V„-' . . . • 'v ■ ■ ■. • • 

. ■■ / ^1 ■ ■ ■ '■■ ■ ■'■ '•■ ■ ■ ■ ■■ ■ " ■ 

Thus. It IS not enough, that, multiple decisigning^ . atterjjatives . ' 

be generated distinct from one. another (mutual exclCTsivenss, 

: • input ). Moreover, the individual eptriesiof 'the derived solution 

set must -.^lemonstrate. such .distinction exclusiyeness, . •' 

: ^iputput). ■ ' ':„.. /'■■■■■ ; \--^-'' ''^:s'y\^ ■ : 
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VALIDATION OF. THE MULTIPLE ALTERNATIVES MODEL , 

As the final secti'on of Chapter four and prior to commencing 
,the development and implementation of the sample models to follow 
in Chapter eleven, it would be remiss to ignore^ a most^critical 
issue of the MAM framework; tht is, "Does MAM do what is purports 
to- do — ;the manner in which 1t is supposed to?". Such a demand 
for ' account abi 1 ity can Only.be responded to with a most hump\e ♦ 
(though gratifying) 'yes. * 

' ■ ■ I ■ 

Model Validity and Reliability Testing 

Validat>on of the Multiple Alternatives Model can occur only 
through its ability to predict performances other than perfor- 
mance in itself ( viz . validity), and the degree to which modeling 
results (the solution vector) are consistent with purported per- 
formance ( viz . reliability). Validating the execution and 
results of a mathematical decision'ing model trius extends beyond 
the simple notion^ ^(5f measurement. In effect, the modeling pro- . 
cess must demonstrate at a minimum : . . ^ 

(1) that the decision arrived at is indicative of the cri- 
teria used in the rendering of that,. decision; and ' 

(2) that the criteria as a collective whole are "predictive" 
- of the resulting decision. 

For validity testing of the MAM system, the distributional 
characteristics of each criterion variable must be analyzed, to 
determine if a significant different exists between teh distribu- 
tion of the criterion measuring the "selected" alternatives and ' 
the distribution representing the "non-selected" alternatives. 



f. 



If the criterion measures areJnterval or ratio- scaled, the^nal- 
ysis technique is simjrly a ^Qn'a^oy analysis of varTanc'e procedure^t^ 
where the independent variable,, Represents whether t^e alternative 
was selected (=1) or not {,rOl? the results of this ANffVA pfeoce- 
dure would determine if the, partitioned criterion distribiAion 
^^representing the selected solfytion. altennati ves actually 
reflected the initial intent of the criterion constraint focus. 
If the criterion measures are nomina]. or ordinal scalecH the use 
of the Chi -Squared procedure (with a* (0,1) independent variable!' * 
is reconmended.- " [' T * 

Reliability testing offers a new dimerf'sion to th!®?'^'alidation * 
of decisioning models; that is, the test of the naocfeled * 
"criteria's ability tb predict future ."choices" between. multiple , 
alternatives based upon established criteria interrelationships. ' 
Thus in the results of a five crlt^jci^r model , we ask the^ 
question: ' . ^ ■ " . ^.^ ' 

"To what extent tb relati^ships between the cr^t^Srion 
variables (constraints) 'exist. In order to permit a ^ 
prediction of future so] ujfe^n decisions based solely 
upon these'^same inter-relationshlpaSi/* * 5^ 



Instead of a single dependent (criterion) aaB Independent .li* 
(decision) variable(s), we are now (In one example) confronted 
with a dichotomous dependent variable^ (0, 1; to chepse^.^pr not) arfd 
five independent variables attempting 'simultaneously to explain a. 
correspondence between select>oa and criterion cijues. This is 
(obviously?). "the protocol for employing ^discriminant functions 
analysis . Applying discriminant analysis, prediction equatiorfS 
(linear combinations) are developed to allow future decisioiCs 
based 'upon the model's use of the current constraint criteria. 

Val idity testing thus* reflects the extent to which the MAM 
solution vpetor reflects an appropriate partitioning of each of, 
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the vector's formulation. Secondly, reliability analysis 
,i,llustrates the degree to which the criteria relationships are so 
well defined 85 to Ije predictive (collectively) of solution vec- 
tor inclusion versus exclusion. 



Individual vs>^Col lecti ve .Criterion Impact 

^ We have reierated^ many times the superior quality of- the MAM 
-system in controlling for the impact (and influence) of interac- 
tive effects. In the previous ^^ection, a^cilysis /of variance^v 
^ prQcedures were recommended to test the validity of decision 

modeling per individual crit erion. However, the MAM framework - 
. does not exist in a criterion-vacuum, but rather supports a 
^ collective criteiron influence upon decisionmaking. The use of 
.discriminant functions to illustrate this collective influence as 
a measure of reliabil ity, is consistent with the focus and impor- 
*tance of interactive-effects modeling. 

■"•^ A more detailed investigation is, required. of the interactive 
eff.ects by the multiple competing criteria because of the addi- 
tional rel=4ance. upon, c^clicJal optimization and the resulting com- 
posite-.cj(9nerated solution vector. Only through the use^of 
predictive linear combinations, can the true "predictable." cri- 
terion impact be understoo^d. 

... ^ ■ <V ^ ' :^ . ■ 

Coiniparative Effects from Maiti-Effects Modeling V,- • 

:# ^ • . ■ V - ' ■ . 

An earlier* discussion of the use of the composite varivable 
rank^tng-CCVR) technique for analyzing potenti al solution 
al|l^^ernat4ves, irftroduaed the main-effects modeling approach to " 
rrfllJ^tip^ alternatives. The results frpm thet:VR procedure can be 
^irectly compared'^to.^thg MAM resylts for overlap, indirectly. 
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the CVR solution vector can be utilized as the independent 
•variable to an ANOVA procedure, or as the dependent variable to 
discriminant functions. The resulting statistics can^then be 
matched against the same statistics resulting from an analysis of 
MAM results; and the "differential impact" per criterion, and 
collectively across all criteria compared. 

The reader is cautioned however to remember that the CVR 
approach does not control for interactive effects. Thus any par- 
ticular choice (solution) takes into account only the mearsures 
of the criterion analyzed individual l.ly. It is reasonable to 
expect then, that CVR results may produce more aggreeable ANOVA 
comparisons. Jhe "proof of the pudding" however will lie in the 
matching of the results from the use of discriminant functions, 
when all teria all taken into account simultaneously (which is 
also realMty*s demand). ' • 

■} - 

Non-Negotiable Solution Alternatives 

You might ask, why go to all this trouble in order to analyze 
a multiple alternatives situation? Clearly, a great deal of 
effort is required to define the criteria, measure and input them 
to -the model. Is it all worth it? And if so, why ? 

An organization's vague, ambiguous* (and otherwise 'wishy- 
washy") approach to decisioning might lead, the casual observer to 
believe that the resulting decision can likewise be non-specific. 
While this may often occur, the actions derived from the deci- 
sions will pot themse^lves j?e vague (aUhough they may be 
incoherent). And the results of these actions will not be ambi- 
guous (though they may provide worthless or even disastrous). 
Decision making, especially in the multipWwternatives arena, ' 
is neither as easy nor as simple (non-comp/ex) as some people 
lead others to believe. 
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Multiple alternatives decision-making can only be rationally 
accomplished via the use of defined criteria; and the conscious 
control of the criteria's competitive interaction. If you as ci 
decision maker can accomplish this successfully while discounting 
the MAM framework, we are most humply impressed. However, if' 
having read the previous pages you can now recognize the 
complexity of multiple alternatives decisioning, and the require- 
ment for a structured framework in which^ to evaluate these alter- 
natives — we rest our case! 
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PART II 



PRESENTATION OF A FULL FORMULATION 
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THE CONTEXT 



;The evaluation of instructional objectives, avai lable soft- 
• . -warei.fdp instructional implementation, and compatible hardware 
uni|sV.'r.ank among some of the more complex decisional problems 
fad rig' "educational professionals today. Like the issues of 
sel.pcting school sites for closure and determining potential 
^r,. program units for roll-back based upon declining enrollments and 
' respectively, a highly structured and . 

; .v:''-^1?em,e'ditated evaluation technique is required in the^reliable 
/'assessment of valid criteria for determining which of the vast 
^ arr.ay;'of instructional objectives are best addressed via which 
r packages, on what microcomputer systems. We will, explore this 
ev^aluation-decisional environment more closely before presenting 
- , a, means for resolving the problems associated with matching the 
/ 'dBsired instructional design to computerized hardwar^e and soft- 
\^ ware components. 



Introduction > ^. / 

* ■ ' The use of data processing techno;,]^^. w;5'?hin the educational 
;dbmain has over the past several years 'Vestri^eted itself almost 
totally to such database management efforts as maintaining stu-. 
/dent and teacher personnel files, purchasing and inventory 
control, and other accounting/bookkeeping' activities. More 
.recently, and with the advent of affordable microcomputers for 
individual building-leVel use, electronic data. processing activi- 
ties have taken a firm hold of the InstructionaT'realm of the 
educational enterprise. .This has been especially true in the 
such classroom-oriented activities as compater-assis'ted instruc- 
tion (CAI) and computer-managed instructioh (CMJ), ■ . ; 
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Acting as th6 classroom teacher's adjutant, the microcom- 
puter provides the hardware" (electronic. "equipm^^^^ arid softwai'e 
;| actual instructional program materials) components necessary to 
promote a rea^nably valid and reliable relay of information to 
the user studejit for the purposes of instruction, drfll and eval- 
uation (CAI)., With additional sophisticated software, this 
electronic "right-hand person" is also able to track the perfor- 
mance and progress of each individual student, compare that 
progress with both local as. well as prescribed norms, and sche- 
dule each student for either remedial, normal or enriched 
instructional activities based upon as . assessment of ,the . 
student's performance -(CMI) , 

Since the overall cost of possessing an integral micro- 
system^has become more reasonable 'over the past tv/o-years, such 
standalone microcomputers as distriDuted by the Apple 
Corporation, Tandy Corporation (Radio Shack), Ohio Scientific, 
Texas Instruments, Pet-Cormipdore, and Health Kit, are becoming as 
corrmonplace as the standard ten-key adding machine w^is some few ' 
decades ago,, And with the onslaught of ^ hardware machines, has;' 
also. come the proliferation 'of 'ready-made* software programs and 
packages available for use within thfe each particular system 
being marketed. Until recently, software dfesigned for one system 
has been^unusable on another system; and thus, "^selection of one 
particular ^microprocessor brought with it the forced acceptance 
^of the philosophy, goals and related activities of the software 
supported by the particular operating system involved. 

Today however,; the days of system-restricted software are 
numbered, with software materials being coded- for accessibi lity 
to^many of the more popular hardware modejks on the market. And, 
as were once the textbook publishers.;cpn(/erned with' usable work- 
book materials to complement their maj^r instructional texts,' 
their same research and development energies are now directed 
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toward designing' micro-software cpmpatible with majpr 
systems, jind paralTe't "to^hei^^ popular text-seri 
able also recently, are diverse' coded packages for use in the,.^ 
adminstrati ve arena of the school setting. . Software packages, v. 1 
designed to perform such school management applications as ^ ' ■ 
salary planning, student data recording, property management, 
accounting, payroll,, personnel data recording; reporting 
and mailing label generation, are now available to 'the pr;incipat . / 
as readi ly as CAI and CMI packages are to the classroom teacher. ; 

With the initial introduction of microprocessors on the " . 
educational scene, th'e5 more logical decisional sequence for se- 
lecting a machine remained ^in determining the utility of^ /: : 
available software first, and then the parallel utijizabi-l.ity of • 
the hardware compatible with the software chosen. Many schools 
nevertheless chose a reciprocal course of action — that' is, . 
purchasisd a hardware unit for whatever reason, and then reviewed- 
the availability of appropriate softvyare for instructional anct . 
management activities — unfortunately discovering- that the more ; 
readily accessible machine was useless unless in^house software ^> ; 
could be developed using one of the cdmpiler languages; and also ;. 
- firii^^^^^^ if any^ school personnel had the training or 

• ability to program the required application(s). - . ; 

The emerging wide-spread availab^Mity of software packages.^:* 
compatible to many "of the more popular hardware systems. oh 
tod'ay's mark^, precludes many of the limitations involved in the 
.'chichen and egg' controversy illustrated.above. However, the .. . 
sophistication and regimen of today's h.ardwaire-software . decisions 
are no less complex or complicated based upon the diversity and 
versati lity associated with the * software . compatibi 1 i ty and hard- 
ware' accessibility. In fact, the decisioning structure could be 
said to noW; be more complex, since such a wide diversity of J 
potential choices — mixes and matches— are possible in .the ' ^ ; , 
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.'f4nal desig"of a /school-based data^ processing system uti lizing 



The NiAA Situation • 

r The;optimal choices associated with matching existing/ and/or 

curricular objectives. and instructional activfties; . - • 
with" a^afilable CAI/CMI software, and the array of microcomputer - ' 
•iystenis compatible with the useable software — exists as one of 
the-niore complicated applications for which the -uti ligation of ■ ' ' 
the MULTIPLE ALTERNATIVES MODEL (Wholeben, 198pa) is specifically . , 
suited! Such a multiple alternatives analysis (MAA) situation 
is really 'a combination of six underlying sub-decislonal systems 
which integrally represent_^ the mix-match solution required. . 
these sub-decisional systems can M;. defined asf ' ■ ' 

tl3 the curriculum subsystem' ~ , that is, assessing : 
the differentia! strengths of'5/arious /instructional 
activities in providing the foundation for valid \ : / 
" satisfaction of curricular goals -and objectives; - y 
and' the ultimate accomplishment of the specific 
CQ^ept learning desfred; 

[23.7 the.' program software subsystem --^ that ;is^!,*^ 

assessing the differential utilities associated . ■ V ^ 
• with each of the available instructional^ 

CM I packages ijijDromoting the • instructional^ ^ ^ V 
. ' activities underlying the purported design and 
" , development of each individual software unit, V 

and its emphasis upon concept intrpdiJction, " p ■ 

• . : activity drill and practice, -'arid assessment ^ 
■ '. .of learning which results';- ' 
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[33 the hardware machine subsystem — that is, . 

assessing the differential utilities associated 
with each of the available microcomputer systems, 

. and their concurrent support of such required 

peripherals as -CRTs, printers, disk .^storage' units, 
central memoryvcapacity, graphic plotters, and :- 
interfacing potential farger, mainframe systems; 

[4] the curriculum/software subsystem. ' — ^ that is, 
assessing the degree or extent of capability in 
, matching some subset of the instructional goals ' 
(activities, obj.ectives) with defjned character- 
istic;^, of software ..packages,^ and the-ultimate 
accomplishment of the specific concept learni-ng 
required for 'normed* performance and progress; 

[53 the software/hardware subsystem — that is, 

assessing the degree or extent of capability in : 
matching some subset of the avai lability software 
packages to the operating characteristics of th^ 

* . various hardware systems, and assuring that the 
* program {software' iJfiTts defined will be compatible 

. to the hardware units selected;; and 

"£63' the curriculum/software/hardware subsystem -t*:.;- 
that is, assesslW total instructional system 
"impact ass del ate^;fe^ 'mixes and . * ' 

V iniatchesVofvthel^^ decisional system^ , 

incorporated Wi;t}iTjp:;^t^ 
' analysis setting. 7 ' .-^ 



• Thus within each of the three ma^or syste^ ' related, to 
curriculum, software and hardware. ind:i vidua! ly, there exists 
sub-MAA'^odel inherent to the overall multiple alternatives 
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■: decision to be made. We will not dwelif upon the obvious, but to: 
illustrate the concept of multiple alternatives analysis, and its 
•. ..re.Tiable.,,,(|ieans. in modeling this CAI-related decisional situation. 

. . . . ■ ■ ' 

. W the curriculum subsystem, the multiple alternatives 
,--are '^Jefined by^the various 'altei^riatTve' activitves- which might .: 
be executed, lio satisfy stated instructional objectives; an'd iii 
turn, the 'alternative' objectives which might "be: satisfied in 

*■ ■ ■ ' ■ ■' f . ••'4 .. . . . .•»'^ ■ • . .-. . 

order to bring .about the desired conceptual .'learning. ■ The -mix/ 
match of potential solutions to this d'elimma. is illustrated by . 
the vai*ious combinations some activities may fbrm. with, other 
activities in satisfying Hhe ultimate conceptual, learning ■ 
demanded of the Instructional subject area or grade-level defined. 

.The software' subsystem: provides a different form of mixing 
. and matching for final solutibfi^ since different -software packages 
may or" may not complei^nt each other — but do portray Varying 
-measures of effectiveness, efficiency, satisfaction .and cost which 
are. internal to the indi,vidual ' packages themselves, thus while 
a :.particLi Tar package may in fact- promote rapid and effective . 
1^ear*ning, the rcost .of this same package may be an ultimate factor 
, in . precluding ttie software unit from forming part of the- solution. 
' ' " V ^ - ; '' • * ■ ■ ■' " ■ : ' ^ ^■■■'^: -,. 

■^;f^na1*^xanipl'e^5^ the ap^l icabi.l ity of MM and its uti.) ity 
'in mode,ll)lg thes^e CAI requir.g^^^ can be witnessed' within the. :, 
a'soltware/hardwarre^^^ the cpmpatibi l-'ity of each . 

..individual software .giiffef or '.the. particular .hardware • (operating) 
•■ system. Intruded W.a^^^^ is controlled for. As 

S^.was a ^rrfnorf mistake _some. few years ago, the model for evaluating . 
the mulli^Tpl 6: alternative^ ^nvolyed in choosing .th^, best, match 
hietweeri d^ired^lea^ avai I able software pack^^^^^ 

and cbmpati-ble hardw^^ must certajn, assure any, selected 

software program ^i 11/ be func^^^^^^ system 
purchased. ' ^ ' > : / , ' » 'V ■ 



Initial Assumptions ■ . . ^ 

As /with 'alt modeling some aspect of a ^ 

deci si ona] environment 'or miTf eu is to. b6 :sirhul ated ( i .e, tested 
for potenti al impact, baied .upon expected oqcurr.enGe^^^^^^ th^re 
exist some basal assumptiGns; Which the modeler muil^^^^ 
be: permitted to acknowledge in-t the final development of t^^^^ 
decisional model. For- the MICROPiK model / key .assu^ 
^'invaive the availability of (and/pr accessibility 
criterion* measures for; comparing the various' subsystem, mixes of 
instr-uctional jicti vaties, software-'. and hardware, the degree to . 
which the. classroom 'teachers- Wi 11. f o-def inning their courses 

• and subject; matter^ into specif icv dif^ instructional . 
Vuni.ts . (observable activities);, and^tft^ extent to^'w different^ '* 

instructional disciplines' (mat.hem^^^^ arts, sfcience;, 

ihdustri.al arts; healttf^ education, . etc.)" can..be co-terminously , ; 
. model (together) ; 

: ;fThe first* assumption - thf avai fabi lif^, of val'id and nelia,- 
ble criterion 'measures^-suitable for evatu^ti ve cornparison ~ i.s ; 
inte'gral to. Resign of the MAA .modeling, framework; and: therefore ; 
a sine qua non requirement for continuation with- further .model r* - 
construction. However; these measures do ■.not-:have :tp 6){Tst' in 
the a priori .^sense.^^^ model des^ign, Jbut^g' cpujrse. mu 
, ble f or successful.^ modeling execution: 

• Such, darta gathering yeq ihvolye.a quasi-'^exp.erirnental 
situation, in which, me.asures.-of effectiveness', etc;/, are collected 
based upon observed (or' rperceived');vper^^rmance'.-— 5^4^ — ^ 
the criterion measures rel ated:'to software ;and.h.ar,elWare w^* 1.1 -have 
tO: be assessed; by^ the' model builder,, a. related assumption es^ists 
Wat the number of software pack a^^^ hardwar?e -units for -the 
intended modeling evaluation- be .l|piited to a. set or Itkelj^ candi.- . 
dates; and .thii? redu'qe the ' necessary compl^x'it^ of ^^^^ to : 
be constructed. :^ ■ * ' ; "/ / "^ ! * ' ■ ' 



The second assumption, and often-the most difficult to 
•realize, is the delineation of instructional concepts and^ goals 
into a finite setof observable and performance-related instruc- 
tional activities. Although the r^rit rebirth of demanded 
specil^city and measured accountability for the classroom teacher 
via such implements as the student learning objective .(SLO) in 
assuring the perforfnance output associated with learning, many 
teachers seem relu<:tant to specifically identify which activites 
are definitively associ^MHA^ desired learning outcomes. 

Over thet decades, the evolved through such 

rhetoric as academij:.-f r^aSw instructional agtonomy into a 
'not-to-be questioned' professional, with an internal code of 
ethics but without the presence of an external monitor. The 
•collapse of the yearly teacher evaluation into a 20-minute 
observation of classroom tactics; and the absence of in-service 
■^instruction for improving the performance of the "experienced 
and tenured" staff person — point to many of the failings of 
the educational domain as a managed and cbntralje^ environment. 

To successfully model the evaluation of instruction^il activ- 
ities against available software and hardware components however, 
requires that such a-delineated framework of instructional ob- 
jectives exist. Again, such delineation does not need to be in 
existence at the commencement of modeling construction — and may 
proceed as the remaining parts of the model are developed. ' ' 

> The third and final key assumption on the part of the modeler 
^^as this CAI-relate(;l model is constructed, remains the extent to 
which the total instructional system-^h* .e. all disciplines) are 
modeled within the same formulation. For most purposes, it will 
be necessary (and acceptable) to model each discipline | 
separately; and thus not constrain the decision^il solution to 
be a resolution compatible to all aspects of /tbe^ instructional 
milieu. This has many advantages^^as^wel 1 as disadvantages; but / 
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remains a more workable format, and one which can be more easily 
/descriptive of the particular discipl inary^.area, 

4r ' ' ' **** ■ 

■'^ ■ ■ 

Pro jectgd ,, Expectat ions 

' ^ ' 

The MICROPIK modeler is cautioned to remember, that the^ 
resulting critqjjion-ref eregced^ simulation of selecting the most 
appropriate software and hardware mix for optimal satisfaction 
of pre-stat^d instructional objectives and "pre-defined instruc- 
'tional activities — dij(Ei,<^to the complexity of its structure, and 
the n^Ve f atae validity givjj|; its processes — will often le.ad 
the .general population (adminis^atQrs and teachers) to believe 
its results (I.e. decisions of r^^tch) as the "gospel according 
to MAA•^ Although this author certainly does not discourage'- such 
disulpleship, it Ms reasonable and prudent to understand^the out- 
put of the CAI-MAM designed system as the best-li|cely decision 
basediioipon the criteria defined, and the modeling formulation 
constructed. ^ . 

Oftentimes, certain specific requirements of a particular, 
^ decision will not (or can not) be sufficiently modeled (that is, 
incorporated within the decisialJj^model design). If the modeler 
recognizes this fact, no compromi^^e to the system is realized. 
.However, the expectations of individuals effected by their 

undeVstanding (albeit rudiipentary) of the modeling framework will 
^ often be impacted by such a conscious (or unconscious) omission. 
Many criterion references may have to be applied to the formation 
of the final solution after surveying the results of the model's ' 
execution based upon the criteria input. Such a subjective 
addition to an otherwise 'objective' modeT'is not compromising 
to the model, V as long as the subjectfve criteria is agreed-upon ^ 
as valid input to. the final decision; and as long as such.additiv 
processes are consistent and visible for examination. 
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. The classroom teacher 1n particular, must, be* brought to 
understand the decision model as a 'best' match''|iDt.. multiple 
alternatives. Teachers, are often hesitant 'to. adopt or accept a 
decision' which is not 'perfect' — and therefore have some diffi- 
culty in accepting the idea of optimality in pr;6bT"em resolution. 
Nothing has provided more of a barrier to the arfbpt>fon of ' CAI anil 
microcomputers within the instructional setting, 'than exactly; Jf, 

this feeling of CAI being *not as good as'-^the f lesh-and-bldbjtfc^^^^^^^^ 

' ' * ■ '.♦f'jj' ■■ 

teacher — and therefore additional -expenditures shfiuldrbe ^^^^^^^^^^^^ 
directed towards greater teacher recruitment and concomitar^t^^^i^ 
reduction of teacher-pupil ratios, rather than the aequisitidf £of 
microcomputers and packaged software. 



Expected Difficulties 

Several barriers and/or pitfalls can be expected during the 
initial design and formulation of the decisioning model, and 
during the examination of its output (modeled decisions). Some 
of these are model-related while others are user-related, and \- 
have been alluded to earlier in this paper. 



The major, and probably most 'key' problem to be overcome by 
the modeler for acceptance of the MICROPIK framework, refers to 
the use of <juantifi able measures (i.e. numbers 1* for measuring 
everything from effectiveness through perceived satisfaction, and 
required revenue expenditure. Mathematicians^^have;'^!^ since 
givten up on the critics who having^ c lajrrie j^ everything 
can be related to numbers', proceed to mainl^n that (therefore) 
"nothing should be'. However, e,ach nlodelin^ituation will not 
be devoid of such criticism, nor will any ajMsptable. response or 
retort be useful. Obviously, all things caW^-^ot be. modeled in a 
quantitative sense — but those that can, should not be ignored 
because of the conflict which may arise. ^ Valid referenaing and 

■■ ■ . . ^ ^ ■ ■ k 
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scaling of criteria, and their reliable measurement — are: the 
best (optimal) defense to the numbers-critic. 

Other difficulties have been referenced in preceding sections 
including the reluctance of teachers to definitively specify the 
relationships of activities to' concepts learned (and objectives 
satisfied), the perception of compromise based upon optimization,, 
and the acceptance, of model ing-by-di,scipline r'^her than including 
the full needs and demands of the school setting — although this 
last problem can often be a strong factor in the acceptance of 
results on the part of thfe individual disciplines or subject areas 

An additional difficulty to be faced by the modeler. will 
concern itself with the concept of 'collective exhausti veness ' 
regarding the inclusion of criteria impacting the final solution 
or decision. It is a favorit-e technique of the modeling critic 
to announce, "... but what if this particular criterion had been 
included in the final design of the solution ... woulcj^ a ^ 
different decision have necessarily resulted?". The simulation 
design must be ready to incorporate additional criteriaVor fe- 
execution of the original modeled framework;, and thereby be ^ble 
to detect any differential solution formulation based lipon the ^ 
existence of new criterion measures. And at times, the mod^er 
must also be ready to state categorically, and be ready to defend 
the position, 'enough is enough'. • » 

e 

A final major difficulty to be faced by the modeler and the ' 
eventual acceptance of modeling results will concern: first*, the 
validity of the criteria selected for impacted and constructing 
the solution, and the parallel validity of the references 
(sources) defined as producing these measures; and second, ^the 
reliabi I ity- of the procedures utilized in gaining tbese required 
measures. Data .will sometimes be available via records, other 
times via sljandardized instruments, and sometimes only through 
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the administration of a subjectively-based 'opinion questionnaire* 
Advance" planning and careful implementation of the data gathering 
portion of the model building sequence, will have great rewards 
in the end^ In^the same vein, nothing so completely nul lifies 
and destroys-^r an otherwise careful modeling effort, than the 
'inclusion of invalid criteria or use of unreliable measurement 
technfques. Even, though rectified, 'the subsequent results^of 
the modeling .solution will be viewed with distrust, and non- \- 
acceptance. ^ ^ * . ' V 



Before proeeedin'g'tci demonstra^ construction' of 

e MICROP'IK modjg^ -for/eyalu^ yaridus .s.o^ and. hardware 
cl^ges acr'gss desireable^.:CiJrr1eular =and ins-tructional objectives, 
^is>iecessa^y to examin#^t(ie?^ -multiple , 

.a:lciSS»ves \ mc^^ in; great er,^t 

ialv/* 3f|iri}age' i^n^ put^'^rernonstrati ve exists as 

■to .the*; deveiq^^ ' implerhe'ntatiqn : of ■ the. 

■.Ghoiqe -altei^atiyes ari^, definition ' 
fts h ips , ■ the ' i ntlusion^ artd , spepi f i c • ref 6r enci ng'. ■ 
the exclusi*^;of ^othef/Ov :an , 
%^Sd.vt^y' WHaiL we i^i 1 l'';porne t^ caTT^'the'.^R^^^^^^ 



th9^x:onti^ol*^^._ , , 

{the^^. '}^iglrA^^ ;.1;o ■e>cecute.;i/n '"a 

; .miiaiier^ 'elMiej^ t^ntj w^^K^tlTje^ or 
s 3(1 cqiTipjiianG^^^^ jnd^de by pp;i:icy;;bQ 



the] 
of 'I 



:ain' 



■ ,-Beffl^ the si^cif ic.:f AI-MAM/m 

.li^ ,1^^^^^ fas h\6n, begin t^> view th^- tecbriT.cal ; wprlcxn^^ of the . 
. .modeling* f'ra^^ how It; jDeff orms th^- .intended eyfiiTuative, ' • 

Gomp?[rASjon;.^a^ ^elecjpan-^^of- alternatives fuhctTpn.;; ;; . : . 



. Introductjtiri/ 





he cQprie)^^ of mul tiple^ a2^ej;'natives dec.vs^ 
f^li^Q' str.a^^K to.:. th^!"* educati onjil ajnalys t.^^ ecti on of.- some 



nuTTiber?- afrJWhdo?^^ large pfloT ' of potential can- 



\Y^dat6s f6r''l:iQisur^^^^ site repre- 

• i^nts/ varyi ng *^easuil^s ' of eff ecti^'veness , i^f Icjen.c^ satisfaction 
• and|S|<pend|ture for each of a "number of criterion references 
(e.^^/- cap^i^jOfj^ h^atip^ requirements, building age. 



\ .,, '.projected- :enrol lment change.,, over 'futiir^,,^^^^^ f |ctors lOf 

■ ;■ ■neighborhood, and proximity of \ptHen. ; sqh6^^^^^^^ ity'; to 

■•^b^ort. trarVsfereeS -in the event^qf: the vf^^ : 
:-.Stotrt^ bl^-t^ese measbr will be adjudgfed: sit^ts^^ ^ 

•;; rtphsatisfactpry) to varying degrees^ cornp^ratile with 

• ■•■■bther sch(Q!Ols'- acrbsi :t districtv. ' v ^ ''vW'^■^% ,^ } ' ' ' ' ; • 

However, vto inclu:% one site f onv.cTp^^^^ 
. an other, site-jneansv^ thatv''gobd'' asp^c^^^ ' 

-^ooT ^must b?- vsacr^ced;^ order lip^^ schqol 
:.vv;:OP)^rational, ev^n ;tho\^^ may Have' 

-^c^Hain unS^tisf acJ:ory-hiecisuV^is^^^^p^^^^^ variables 
V ^whi-cfV^ the now clo$ed schdj^l . e,j<:.6^^^^^ 

^ ;'"o|e,'l;i t^vcrf -t^ is^'1<noi^n as interactive 

« 6|fectS'^nt6'd§^^^^^^^^^^^^ the necessity 

bfitriSfetruc^^ng ,$olut;ipns;-.,i^^^ include sbme 

'form of ;^.oriirol>l^fe^l-:p^^^^ as the' 

i^;,., var^ jjius. alf^ are ;^va.luaiieaK:'^ 



■ ■ J ■i;*!.W>-v,ri • ■■■ 



7 ■" 'The Issue oficonjiDt^^ipjpif^i"^^ statement 
f \ of .the prob^^ t|:;;seJ_e^Wf^^s^^^^ number ,'|f schools, fbr closure in 
1^ "brd^er - to promote c^^rtiifir-d^^^^^^^^^ the district; and thus 

1^ ;^to determii^ how many, s-chools wil^J.-b*^ closed and which ones. 
^ Obvious^y,^ be simultaneously per- . 

M f(r^^4^>p}^se: twd/.int^|^el ated ^^ci si ons : "how many?" and 
^ ."wh^Jcmes^^^ ■ ■ . . ^. _ ' : ' 

■ ' " ^ ' '^^ '''''' 

, determijiation of which program unit budgets wi H be 

.^^deersioned^'.for cojiH^ed fiQrrding {versus deallocation) is another 



exampl^e of ||p* multiple alternatives framework, arid its superior 
Contri^^on to the r^e'hlm of accountable and criterionref erenced 
•:»>eyalu^tiOT ^a^^ (Wholeben and SuUivafi< 1981). In 

the f^c^al dsal location model, criteria represent the projected 
\'ex5endituref witVin each object cost code for each of the units 
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under evaluatiort; and in addition^ contain perceptual measures of 
administrative level of expehdabi I Ity. Once again of course, 
exists the dual responsibilities for determfning how many program 
budgets will be discanti.nued,. and whicfr ones — based upon the 
interactive modeling effects-: of the various criterion weights . 
across unit alternatives. 

The Criterion Vectors 

• The. multiple alternatives. .made-l js:-.silTip-l7---a-^ of 
simultaneous linear inequalities "and-^equalities which collec- 
tively represents the problem to be solved. Such an algebraic 
linear system -is portrayed in <Figure 1?. Note Jjow each linear 
combination represents a vector of values^( vt^< % coefficients) 
which identifies the total, measureable/impact to a system of the 
alternatives- being modeled. Thus there exists a unique 
(normally) combination of coefficients for each of the criterion 
references used as input to tjie decisionirra process. The alter- 
natives themselves are further defined as binary ^ariables (that 
is,, taking on the value of either 0.<or 1 (to be excluded in" the 
final solution set, . or to be'included; respectively). Vector 
formulation for each criterion reference, 

C a^xi ai2X2 3^3x3 ... a^-jXj 3 

portraying criterion references across ^ alternatives, will 

then provide a basis' for measuring total impact to the system as 

a whole attributable to the solution set constructed. Bounds (or 

limits) to what is allowable/^a total impact to the system are 

expressed as vector entries^'ithin the conditional vector (or 

■■■■•■■§ *■ ■ 

normally named; RHS^ the right-hand-side). The,RHS-values are 

• ?■/ 

the, constants of the^equations ^^ind inequalities modeling the 
system. <Figure-2> presents a listing of the four generic types 
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^''^'^^''''^ ^° Which each model should address content validity; 
and <Figure 3> depictsjbhese criterion entries as members of the 
.modeling framework pfeviousl^ i 1 lustrated' within Figure 1. 



The Objective or Optimal:i,ty - y.ector 



The remainder of the modeling process concerns„.the use of an 
additional vector to assist in determining /rom th'e potentially 
hundreds, (or millions," in sqme .exercises ['of possible 
alternatives, that one, best mix for which the' best, ^possible 
solution exists. This process, is calfed the Search for ^ 
optimality, and the vector is known- as! the objective function (or 
sometimes, the cost vector). Gepmetrically, the objective func- 
tion is, a n-l.dimensional figure passing through the n-tuple 
space (convex), which is feasible (that is,, includes all' of the 
constraints postulated through the use of the linear equalities, 
and inequalities) and which seeks a minimum point within'% 
feasible region (if the goal is to minimize the impact of ,th^^ 
objective function's values upon the system) or a maximum potnt 
vyithin the feasible region (if the goal is to maximize the 
.defined objective function's impact to the system as a whole). 



The Goal, of MAA and MAM ■ ' ' 

■^-'Simply stated, the multiple alternatives model is a tech- ' > 
nique' which seeks to construct a solution set (a vector oBl's " 
and O's), such that this same solution vector rep,r;-esents the ■ 
solution ofSthe simultaneous system, constraihed^^/a series of. 
competing critgrio.n measures (vectors), ind based upon the ooti- 
mality demandsj of. the Objective function. 
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THE. MISSION 



V 



- ' ■ As with all complex applications, of planning, design 
: development' in the construction of systematic evaluatipn and . 
^?.;:];decis ion-making models, the MJCROPIK framework is built,, upon a 
v;^ieTineative deductive base. The overall goal or mission, of the 

MICROPIK model is to^jj|p|rmulate an evaluation and decisioning, 
■ procedure, based upon the ^criterion-referenced assessment and 
comparison of various optional .alternatives regarding curriculum 
goals, available software and compatible hardware; and^^ to model- 
this evaluation framework as closely as possible (i.e. simulate) 
with the established needs and demands of the school environment 
involved. In a more simple sense, "to do what needs to be done, 
and what the properly ordained decision-makers'wouljj do, if they 
only could". Sounds straight-forward enough, do you riot a^ee? 



Mission, of the MICROPIK 



,It is the ftifssion of the. MICROPIK modeling framework to . 
design and. develop: / ■ 

a multiple-alternatives-,' criterion-referenced 
modeling structure — evaluating; and comparing 
potential microcomputer instructional software ' 
and related machine hardware —resulting in an 
informed decision as to which software packages 
and hardware units are, most optimal ly suited for 
enh,ancing/t.jie established instructional objectives 
for computer-assisted (CAlj and computer-managed 
' (CM!) instruction within '^th^ . educational ehterpris^e. . 

\ . . , » 

A secondary statement of mission iscalso possible, dealing more 
generally with, the CAI-MAM aspect of the modeling framework^ yet 

* - 182 ■ ' ' 
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more'Sp,ec1fical:ly'with the notion of. decision modeling; that is, 
to design and develop: 

a; deci signing simulation structure — capable of 
incorporating the/desired, potential decisioning 
alternatives of the major policy bodies, and the • 
relevant, valid criteria,,admissable to the needed 
' comparison of alternatives — and in f-uJT accord 
with established policy, consistent practice, and 
mandate^legal principles and individual' riglits. . 

While the primary statement of mission (above) deals more directly 
with the 'framework and constructs of the MICROPIK application of 
multiple alternatives analysis (MAA), the secondary mission 
addresses specifically the foundational constructs of the under- 
lying multiple alternatives model (MAM) . itself • 

Major Secondary Goals • :; 

As -with the primary and secondary statements of missfon - 
defined in the preceding section, design and development of the 
MICROPIK modeling framework will encompass severaV del ineati ve 
levels of goals,' objectives, activities .and tasks before the final 
MAM structure is ready for execution. The construction of such 
a systemic model is itself an exercise in implementing the usual 
constructs of a more generic "planning model", A' developmental 
paradigm (roadmap or blueprint, if you wish) is essential for the 
controlled construction of a reliable decisioning technique; ■ and - 
that ^^hnique's inclusion of valid datum and algebraic relations. 

Parallel to the' normal (major) goals which' would accompany 
such model construction (e.g. planning, historigraphic review, . 
general design, field-testing, implementation, and assessment) 
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certain secondary goals are of- dei^nstrati ve Important within the 
modeling episode; and bear illumination and clarification .at this, 
time, , .^ ' 

The first, major secondary goal within the design and devel- 
opment of the MICROPIK framework, is to itaintainvigi lance upon 
the mutual-exclusiveness construct— regarding both alternatives 
included for comparison, and criteria chosen for performing that ■ 
comparison. Alternatives should. be separate and, independent 
(i .e. mutually-exclusive) of other iaTtern^_;tives within the' model. - 
This of course will not 'always, be desirecible; and at times, the 
modeler will seek to* co^'r^elate the usefulness of one alternative 
based upon the parallel ^^istence^^f another alti^rnative. This - 
would especial ly be true of . aji instructional objectives and 
activities model, whjere sequential and progressive learning and 
reinforcement must be available^ for optimal concept learning. 

Parallel vigilance upon the mutual-exclusi.veness of the/<^ 
criteria included within 'tjie model ing^i^framework -is a matter of 
mqdel efficiency, rather than a source of unrelTabiJ ity. As irt 
the. past 'dark history* of evaluation and^ decision.,niodeling, the 
mbdel .builder has/ not always, maintained, the* hig'hest. professional- 
standards: and has tberefdre constructed the^model; to best depict ^ 
;t:he. specif T^decT5 ions desired. This procedure of 'stacking thd' ^. . 
model' is not possible with the MAM framework, in terms of 
inclifding a mass of 'stacked^ criteria to weight intended decisions y 
in a certai^a direction. However, this is a major, concern when 
addressing the construct of'^.criter ion cotlective-exhaustiveness. , 

The next, major secondAry- goal within the design arid- deyeK^^z 
opment of the MICROPIK frarffl^rk:. Is /to 'insure the col lecti ve^ . i 
exhausti veness of both alternatives- compared, and' evalua^ .: 
criteria utilized. Completeness or systemi'Cr, totality of lihe^ jt. . 
diodeled simulation is\ of ^primary imp^ortance; and exists- as one of 



the most potentialjy coniipromisi ng .circun:istances regarding the 
possible nullification d^^^^^^^ 

Without the coTlecti>.e-exhaust1veness of ^'he multiple ; . 
alt«rnatiives, represented within the modey immediate criticism 
will be directed towards the model as not comparing 'all possible' 
decisional alternatives. Al?d, even though some alternatives may 
"^i- 3. Prjori determined to be a. necessary part of the final solu- 
tion (regardless of their .attributes as measured by the criteria),- 
these same alternatives must b& included within the model in order 
; to SHjimarily include the impact- to the system. as a whole, based 
upon- their 'forced' . inclusion within the solution, vector. 

. * • ' ■ . 

As^mentioned' above, the cb.Tlective-exhaustivene'ss associated 

with the criterion-references must be ^, major -concern of the . 

model builder.. Sjmply statted, if a particular criterion is not 

a part of the MAM framework, then- neither its impact upon the .* 

various alternatives involved nor ' its effect upon the system as 

a whol.e can be represented and cpntrolled. Unfortunately, the " ' 

construct of .Qonej:tive-exhau^tiveness applied to' qriteria is 

-also one of the primary nemeses of the modeler.. Without a doubt, , 

demands will exist to include >'new^ and 'different' criterion 

measures in order to survey their resulting impact to the model's 

decisioning process; the "....but, what if ...?" situation has 

been mentioned previously. Reconstruction of the model, and, the ' 

related summary -of new resul ts can be very ^djous, time consuming, 

and moreover nerve-rackir\a for: the .modeler. ' Because, of the time 

and expense (both fiscal ya^ell is mental) involved, the actual 

:1^4§pendenc.e .or Tion-col.linear^"ty of acfd^ criteria can often 

' ■ ' ' ' ■' i ■ • ■ f ■. ■ ■• 

;.b|e addressed via such'^avaiJable techniques 4s parametric or 'non- . 

'■pkrametric bi.-variate cprrelati on methods, and/or the use, of a 

oiheway ana.Ty^i s> of 'V^r i ahce procedure (to assess rel ati ve bi as )^ 

The* thir.d, major secondary goal associated with design and 
devel bprngnt , rpertains to the ref erenci ng, ;seal i ng and measuri ng 
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of these inutuai ly-excTu,s1ve and collecti velyfiexhaustiye criteria,^ 
Oftentimes, a criterion wi l 1 be defined (e;'g. satisf action); ,;//nich 
defies ^direct,, physical measure, . and m^^^ therefore be referenced 
and -.measured via more;;Synthettc . techriiqu^ 4{e.g.. OQin ' 
surveys) to'- obtain modeling ihput.CWholebenY 1980a^ igsdB^V^ Wholeben- 
arid Sullivan, 1981) .In other cases, the method dfscaling/the . 
sought criterion measure (that is- how quantified), wi ll provide 
declarati(^s of potent*ial uhreli ability frbm\mpdeV critics. For ; ■ 
example, ^witness the ongoing . controverisy concerning -^Ha use of . 
, the. agreement-coritinuum .wherein proponents.' of, ^the^f^e- • 

'.. ■STR0N(3tY " . V ' ■ ' : ''^SiMMu; ^'^^ "'^ 

•■• P^AGREE . DISAGREE- > OPINION . AGI^^^./'A^REB : ■ ; '■ ;■ 

continuum scale ^"strongly disagreeV-vwith th'e ^^ix'-point: staT^:*^ - 

■ : '■ t ■ . -'^i^.-i.,'. : .... i 

■ ' STRONG ■ MODERATE ' " MODERATES/ ; ' . STRTJNG 

DISAGREE .. DISAGREE . DISAGREE . , AGfTEE^^^^^^REB AGREE ^ 

. ■ . . • ; ' ♦ . ■ ■ . . ■ . . ; ■ . ■ . ' ■ ■ " 
whose - proponents state categorica:=lly,.',that "ev^^orie-'has some 

/degree/; of opinion, no matter how' smaJT or, truty/'.fininformed'*; >^ 

■ The controversy .associated with referencing can. be 

often only marginally^defensible by the model builder. For 
^ampli':,- if: you want to\ know .if parents jire\dis;^f?fe^ 
mari^^ement^ ;i nstr uct i on' of their ne.fghborhobd^^.^re ^schdaT^ 
as*' a meafsijre of potential for the: site to bp clo^|^n''aTi era of v : 



declining eriroTlment;--- you may not ^;/ish to' ask the^qjjestion- v4a 
' a survey; "Are you ^||;tsfTed w ydur xhi ldren' s sc^hool?", in a" 
climate of potenti aTi'eflijriiriatT of school 'site^. Other 'backaoor<i 
methods ,wi11 be necessary to^ obtain measures of< satisfaction, 
without pre-biasinjg theJres'pondent*s inputs . " . J 

A final, major 'secondary goal to-^^^ design' 
will concern the po^frle, .desireable- weighting of some criterion ' 



measures over others. Several techniques are possible for this 
within the MAM framework (weighting individual vector entries, 
modifying the RHS-vector, and weighting various •solution vectors 
from the solution tracking matrix of cyclic optimization). Not 
only musj: be valid and reliable technique be utilized in the 
event that weighting 1s necessary; but so also, must the procedure 
for obtaining the direction ^nd extent of these weights from the 
3eyond-rej>roach.^ ' 



Select 



or Milestones 



As iMh all planning activities which include a systematic 
approach tif^sign and development as welt^as a' heavy time comit- 
tment for imj^Pljpientation and evaluation, several 'points of 

-^frtenti-al cuiiuyr^ii' b iiie q'ua non can be identified by the modeler. 
This-points or decision junctures are important in that if any 

"deTay to the activities pre ^ the juncture is experienced^ 
the whole process will be delayed; or. in the parlanc-e of the 
planning and networking theoriest.a 'bottle-neck' formed. For . 
the reader additional understanding of the developmental aspects 
associated with model design and implementation, the "*fal lowing 
list of selected major milestones has been formulatedN^ 



M-01 
M-02 
M-03 
M-04 

Mr05 

M-05 
.M-07 
M-08 
M-09 
M-10 



ACCEPTANCE OF THE MODELING ENVIRONMENT 
REVIEW OF ESTABLI-SHED POLICY/PROCEDURE 
, DEFINITION OF CONTEXTUAL NEED/DEMAND 
"STATEMENT'OF MISSrON/GO;C\LS/OBJECTIVES 
FORMULATION OF ALTERNATIVES (w/ REVIEW) 
DEFINITION/REFERENCE OF CRITERIA, (w/. REVIEW)' 
DATA COLLECTION/SCALING (w/ REVIEW) 
EARLY FIELD-TEST OF MODEL (COMMUNICATED)' ' 
FULL-SCALE EXECUTION OF COMPLETED MODEL 
ANALYSIS AND SUMMaVy OF FINDINGS 
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: VAUIDITY AND RELIABILITY ' TESTING* 
- ^ M-12 : PUBbIC HEARINGS ( w/--F I NAL" REVIEW) . 

■M-13 ": SELL HOME AND LEAVE TOWN' (w/o REVIEW). -^ ^ ' 

This Is hardly^an exhaustive list; and\with even a minor. 



clarification and delineation of .topip 'Could Yesults ^in several 



delii 

hjjndred milestones — each as. important as th^more relevant 12_ ^ 
efxpressed in the' above listing.^ 



Finally, th.e;|!non-planning..theorist * reader\must also under- 



sAand, that the ibb^/e milestones need nof be addr^s^^^(and 
plann'fefl for) in an independent, separate fashion. \ ^Many facets of^' 
the modeling protess take^Jace in parallel order ras opposed to 
serial); and so several phases of the modeling process will -be 

• ongoing .simultaneously.V One of thp best and most i^I Tustrattive 

examples of such simultanejty occurs during the^aTt-ern^^^^^ 

'development phase, (as alternatives are def tned ^and explored, the 
modeTer will find, it hard not to (in parallel) also explore the 
types of criteria which would be useful in evaluating the various^ 
alternatives, how these criteria might be defined, references, 
scaled and measured and even h5w they might be formulated^ 
within a criterion constraint, vector for input into the.MICROPiK 
decisioning model . Of course^ some aspects are truly serial, and 
can not be performed simul taneousTy; for example; the serial 
order of the field-test versus the full-scale^ implementation. " 

• 7 ' ' ' 

We will now examine! in specific detail, the illustration of 

the M/y\ and M.ICROPIK missions, and the implementation of their ' 
j> > ■ . ' 

> stated secondary goals.' ^ 
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CHAPTER 06 



THE DEVELOPMENT STAGE 



. L Constructing Principal Modeling Components 
for* Decision-Making ] 




... THE ALTERNATIVES 

The fins't major phase of MICROPIK d|§S^^M|Mevelopment ' , 
cocerns the identification, -definition andi^^^^rannt of, the. 
multiple alternatives to be evaluated iiy the'-M/^^^[newprk. Tiie, 
reader wii; recall, that the mission of this', MAM^^; mode I'ing' ' 
exercise seeks to evaluate stated curricular objectives ,and 
instructional activities (and their projected ijfi^^luence upon^ • 
desired degree of related concept learni ng),; thj|||ppropriatene"*ss' 
of available CAI/CMI program software for ^mplem^ft^ these 
instructional learning exercises, .and the correlated compabil jty " 
p-f existing microcomputer hardware ^including peripherals) to 
execute the various progr^ifffspftware. packages. We will develop 
the -alternatives-portian 'of t'he MICROPIK modeling framework, with- 
in this current section; and reserve, the next section for an, 
exploration, of the necessary critei^i a to evaluate and compare ^ 
these alternatives. 

• ». 

The reader will/also recal.l , that although such an evaluation 
of curriculum-saftware-hardware alternatives, and. their inter- 
relationships /could very well be an end in itself, the author's"' 
over-riding concern is to posit a decisionipg model'by which 
schpols and service districts will be able to make ^intelligent' 
decisions regarding* the acquisition of computer software and 
hardware, and'.its utility in fulfilling stated comp||jter-assisted 
and computer-managed Instructiohal objectives. 



A Tri-Part ite Hierarchy , , " , , . ' < 

' '. ' - ^ . • ^ ■ 

■ As was demonstrated in theAinitial development of the 
".curriculum activity packaging" -(CAP) model - (Wholeben, 1:980b), 
a MAA modeling of 'c^Lirri'culum 'objecti ves- and instructional 
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aC:|:i.vl'tTes as rel'^ted to concepi; .learning .co.uld be' demonstrated 
via a del ineatTV6' or hTferarchfcal -framework,' Consider the usual '/ 
-representaVon of the concept-;Objectwes,'.act'iv.>'t1-es environ: ' 



Objective 1.01 



CONCEPT l.d - 




1 • 

Objective 1.02 



■1' 



I, 



Ji'y' : I ■ I 1, ■ 1 ■ 

Activity /^'cilvity' Activity 'Activity /Vctivity Activity '• 
,1.01.01, 1.0r.02 1.02.03 1.02.01 1.02.02 1.0C'.03 



' The multiple-alternatTves formulate'd MICROPIK model seeks to • ^ 
satisify to some optimal degree, all concept and objectives- ■ 

• related learning as specified by curriculum requirements. Th^ • 

^ existence 0f-multiple-alternat'u*^r?or MAA evaluation exists iii 
the formication of the various activities "which m'ight" be 
implemented 'in order to meet instructional ( le'arningUneeds arid 
demands. In the MICROPIK setting therefore, ajN concept and 

'objec1:ives learning must,be Satisfied — It remains the activities 
which will evaluated for their relative utility or appropri-ateness 
in fujfilling this required satisfaction. . . , 



• • jki'amore advanced .formulation of the MICROPIK model , where 
objectives are to be' cons i'dere'y- alternatives' available for com- " 
^paritive assessment as well as the underlying activities which 
demonstrate the execution of the objective's intent,' it is sti II 
the evaluation of the. activities which w1 IT not only deijionstrate 
their utility for incTusloh'withln th|, final curriculijhi package, 
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but also >induGti vely determines whether" the particular objective 



which defines their presence will be itself associated with the 
final solution- set . The reader should also see therefore, that 
alternative 'concepts* could also be modeled.in this way. 

' This three-level or tri-partite hierachical formulation -of 
the Multiple-alternative^ structure is extremely useful to the 
modeler, should such defined sophistication become necessary ' 
based upon the situation being simulatecJ. As we wi 1 1* see,^this 
delineative structure within the alternatives definition will 
become one of the major modeling constr'utts to emerge from the 
design af the MICROPIK framework. 



The Sectional Alternatives Vector 

Because the MICRffPIK model seeks to 'evaluate. the corres- 
ponding relationships between curriculum,, software hardware 
— asJwell as compear i^ons within each of these three gr*oups — 
the , structuring of mo^deling alternatives may be classified into 
the three major groups: 

[1] curriculum/niCtruGt-i^nal al ternat;iyes; \ \ \ 

[23' CAI and otfier program" software alternatwes; and 
t33 hardwawS and. peripheral (s) alternatives* 

..." I - ' , . . 7- ■ 

As with the tri-p-artite hierarchical development of the^curricul 

* ' ■ t' ' - 

objectives and instructional xicti v.ities, the design of both the 

software and hardware Alternatives will assume a hierarchical 

setting. 



Structuring the Xpcond sas^^on of the alt^natives vector 
(recall that the first section Refers to the. \cu^ric.ulum entries) 
will be primarily concerned witrN^diff er^n^ aspects ^f the same 
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curricula^ or disciplinary framework ^being modeled in the first 
• section. For example, languag^ arts may, require GAI packages 
which related to various types of instruct ions^i such as: 
reading, spelling, vocabulary, sentence structure, ancj analogies, 
Several software packages may exist for each Qf the above five 
Yequired areas which will summarily require^ evaluation both, in 
terms of their variable values .between each other (package) , and 
in terms -of the'ir utility in addressing the stated instructional 
afctivities. Jhe hierarchical design for this, section of the ^ 
alternatives vector rrftiy be constructed as: • * ^ 

^ - SENTENCE •' 

READING SPELLING VOCABULARY , STRUCTURE ANALOGIES' 

■ - ■ .\ : • J .. I . I ■ • I 
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. i 1 \ I i 1 I r I.I J I 

\ Pkg Pkg Pkg Pkg Pkg Pkg Pkg Pkg Pkg Pkg Pkg Pkg 
/I 2 1 Z -12 3 1 • 2 3 -1 2- 

' " ■> ^ a ' ■ 

. The above seemingly bi-partite design could easily ,aj^sume a morei 
tri-partite status fs difference- grade-levels for CAI within the 
elementary school setting became a new, confo4JUKJing variable for 
modeling within the language* arts^ portion of the MICROPIK model. 

o , ■ ' ■ . ■ ■ ■» • 

. * • ' . ' J. • 

■ X' ■ ■ ■ 

The third and final part of the sectional alternatives vector 
will contain the various multiple alternatives related to the 

'Utility of various hardware machines (and their peripherals) in. 
, implementing the evaluated comparisons between, £he curriculum 
desired, and, the. software packages which best ^instriJct the re- 
lated instructional activities. This particular collection of 
hardware alternatives can be greatly sirtiplified if the modeler 

• ^in advance agrees upon 'hardware packages' f.or inclusion within- 
the MICROPIK formulation. Thus, a certain' model of APPLE (e.g. 
APPLE II PLUS), a certain type of printer, and a certciin number' 
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of disk drives might become the "APPLE" package, and therefore a 
single* alternative for comparlsoft against the "TRS-80" package, 
or the "OSI" package, etc.. In comparison with the tri-partite ,^ 
hierarchical structure of. the instructional activities, and the 
' bi-partite^ structure of the^software alternatives, the Ijardware; 
section of the alternatives vector woulor^ecome a uni-partite 
or single-level collection of multiple alternatives: 

■ , , ■ . -f ■ ■ ■ . : 

' APPiE ; TRS-80 OSI ATARI ^ TI ' 

PACKAGE PACKAGE PACKAGE PACKAGE PACKAGE ... etc. 




However, should different models of the same micro^rocejssor be 
required for alternatives decision-making, and should varying 

* types of peripherals be required for inclusion within thafull' - 
MAM formulation — a tri-partite (manufacturer-model-peripheral) 
relationship reappears. Because some manufacturers have refused 
to keep their software model independent (e.g. some TRS-'BO II 
packa'ges will not work on the TRS-8(> III; arid likewise for the^ 
latest problems between APPLE if PLUS and compatibility with the 

^ APPLE III), a higher-order 'decision may need to be made concerning 
not only' the type'of software and peri-pheral required, but also 
the compatibility of the. 'level of model' needed to execute the 
compatible so'ftware. The discerning reader can easily see how 
a quad- or even quint-partite hierarchi^cal structure may be 
necessitated by guch a complex mul tiple-alternati ves^'setting. 

^Summary ; ^ ^ ' \ . . 

Thus the alternati ves^^vector fbr exposition of the MICROPIK 
model is divided into three main sections: the tri-partite 
curriculum section, 'the bi-partite software section, and finally 
the (hopefully) uni-partite hardware section. However reader 
is cau^tioned regaVding the true partitioning of the hardware 
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section .of the alternatives vector. It is vfery likely in consid- 
,eraJffon of the problems with the lack of upward-cbmpatibi lity of 
-^particular system's software," and indeed in the" quality- 
differential between peripherals and the type of peripheral (e.g. 
graphics plotters), that the hardware- section could easily take 
ort quad-partfte characteristics. 



i llustr 



cr. In summary theriiVthe alternatives vector can be illustrated 
as follows-: < 



j CUf^RICULOM- COMPONENTS I SOFTOARE COMPONENTS I HARDWARE I 

I 1 ..........„...„.„.„ , , 

I ACT ACT ,ACT| ACT ACT..|PKG PKG | PKG PKG PKG..^ PKG PKG PKGv , ^ 
i 1.1 1.2 l.'a j 2.1 2.2 I 1.1 1.2 I 2.1 2.2 2.3 j 1 2 3 | 

L— J -L \ - I 
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THE CRITERIA 

••■ \ To .fulfill the stated premises of -the MICROPIK model in 
determi-ning the appropriate microcomputer hardware" and software 
in terms of stated instructional requirements, the various . » 
sections of multiple alternatives described in the previous topic 
must be evaluated^across various-competing criteria. As was 
mentioned in a previous sect-' on, the MICROPIK decisibning. model 
requires a total of six 'typ^s\ of criterion formulations: three 
to attidres's the intra-relationships existent withtn each, of the 
three sectional areas of curriculum, software and hardware — to 

.allow cross-comparisons of the various alternatives within each., 
of ;ehe ra^ain alternatives* sections. Two. addi'tional criterion 
sets* are required to measure those inter-relationships which will 
need to be corVtrolled between the sections of curriculum versus 
software, and softw^lre versus hardware. It is assumed, thcit the 
fhird' possible bi-sectional criterion set which would relate 
curriculum versus hardware can be based upon the trichotomous ■ 
'inferenc^^B resulting from'the first two bi-sectional comparisons. 
Finally, a criterion set will .be reserved for an overall, tri- 
sectional evaluation of , 'curriculum v. software v. hardware' *. 
inter-relationshif)s. ; 

^ ■ \ ■ « 

Generic Criterion Indices , 

As with all planning, and development activities, the modeler 
^will find the utilization of a 'philosophical' model most helpful 
in identifying and defining 'types' of criteria which may prove 
useful \n discriminating, betv^een ^the multiple ^alternatives. This 
is of great/er yfmportance within the CAI-MAM .framework due to the 
complicated relationships both between the three general alterna- 
' tiyr sections (curriculum,' software and hardware) as well as 
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within each of these gei^eral sections. Before a general, listing 
and discussion of more specific criterion indices which'wi II Te 
of - some benefit to the MICROPIK modeler, .a mor^ genus-oriented 
discussion of criterion^type will be presented. 

Three .categories of generic criterion indices seem to exist* 
for all problems of evaluation and decision-making when dealing 
with^^mul tiple alternatives:. 

^ [Ij index of ^contextual need based. upon* performance; - 
[2] index of relative worth or value; and 
[33 index -of general resource or expenditure. • 

The/ ind^ of contextual need ba"ked upon performance is -itself 
a relative comparison between, the measured. states of perceived 
need, current performance or use, and observed demand. Such con- 
tingenciesj as where demand is greater*than need suggests either 
an unrealistic understanding of the enterprise, or. an equally ' 
unre^istics understanding of the characteristics of whatever is . 
declareHj'in demand*. . Of course, a contingency of need .greater 
than demand might also point to a lack of understanding of the ' 
context within^which the. organization exists. Indicators'^ such as 
might indicate waste (demand greater than performance and/or need) 
or intervention (need greatfer than performance) must also be 
-addressed in some fashion as part of the contextual need set of . 
criteria. 

< ■ ■ 

The index of relative worth or value is often more easily 
modeled into an evaluation framework due to its more 'esoteric' 
-issues of: effectiveness, efficiency and satisfaction. To be 
effective, whatever is performed (or in our case, selected) mj^st ^ 
"dp the job". To be efficient, the selected alterT;fative' solution 
must dp the*^job as quickly as- possible, and within . the stated ^ • 
operational., limits of 'the enterprise (or less). And to be, 
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considered satisfactory, the solufion must portray 'goodVfeelings 
on the part of all parties involved; or^ at the leasts be consistent 
.ini this/regard. 



0 

w < 



: The index of general resource or expenditur-e i s a more 
direct relating of N alternatives tp those elements of capital, 
revenue *and/or expenditure which might be required in the final 
4'mplementation of; the selected alternative. Such resources as: 
time, space, facility, personnel,^ cost,' supplies and materials, 
and equipment — • Wil'l all te a potential part ^f this^ particular 

criterion set, ♦ . - \ 

....J'.' *• 

With these ideas in mind, we. can now move to a more specific 
,develo|3ment of Sample criterion references for the MICROPIK mo^el , 



Identification, and Definltioli ' v „ 

. 'To explore the. vkr^ious criterion indicators which will be of 
use In evaluating the curriculum, - sof'tware and hardware multiple 
alternatives associated with the M^^OPIK framework, an outline 
format wi 1 T be presented foV the reader * s, perusal . This outl ine 
will, examine each of three major alternative, sections first, and 
then examine ^potentia^^ criteria for performing the aforementioned 
b|-sei:ti octal and tri -sectional. comparisons. 



Set i;.0. THE . CURRICUiaM SECTIONAL / ^ 

(examining relationships both within ^curriculum 
: / ' objectives, and between the various, multiple 
instructiorial activities'^ 

. ■ A' ,, " ■ • ^ , . 

■ V - . ' . . • ■ 

1.01 'measures' associ ated with perf or:mance, need and demand 



1.01.0, 



perception of school personnel 
(administrators, teachers, students) 



1.01.02 ' 



observed time spent on classroom instruction 
for ^rvarious. topics,' group instruction versus 
individualized or remedial requirements 



1.01.03 



relative importance of the curriculum unit 
based upon district level syllabus -standards 



1.01.04 relative importance of the curriculum unit 

compared to all other required currlcular units 

.1.02 measures associated with worth or value 

1.02.01 perception of effectiveness, 'efficiency and 
satisfaction on part/ of classroom teachers 
and/students, for each curriculum unit 



1.02.02 




1.02.03 



observed measures of effectiveness as relate 
to learning and retention 



1.02.04 



observed measures of efficiency as relate to 
time required for different instructional 
strategies ' - 



1.02.05 



related utility of each unit for success in 
the adult or occupational world-of-wor(< 
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1.03 measures associated with general resource or expenditure 
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r.03.or related requirements fok equipment, supplies or ' """^ 
other riiaterials^in implementation of each unit 



1.03.02 necessary spac6 aRd/or facility requirements 



1.03..03 related personnel staffing needs 
* ^ 

1.03.04 measure of relative impact upon other programs 
based upon resource . allocation 



1.03.05 related costs and/or expenditures for each unit 
based upon text books, work books, etc. 



Set 2.0 ' the program SOFTWARE SECTIONAL 

(examining relationships between the various software 
packag es available to perform computer-^sisted 
instructional/managerial efforts)' 



2.01 measures relat'ing to the availability of various CAI/CMI 



and other-^dmirristrartw and their 

comparative uti lity 

2.01.01 compiler languages 

2.01.02 word processing 

2.01.03 operating system languages 

2.01.04 data analysis programs 
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2.01.05 database management programs 

2.01.06 management -planning programs. 

^2.01.07 time/project/personnll scheduling progr.ams 

. 2.01.08 accounting software * 

b 

2.01.09 specific CAI/CM^[^^seware packages 




2.01.10 CAI/CMI author pilots 

2.01.11 graphics packages 



2.01.12 system editors 

2.01.13 information retrieval service 
(communications multiplexors) 



2.02 measures ^elating to the results of sample field-tests 
or use by other in3fTviduals, concerning effectiveness 
and efficiency in presentation and drill, ;and related 
satisfaction on part of using parties 



3.0 THE MACHINE ' HARDWARE SECTIONAL 

(examining relatidnships between the variousf-hardware 
' ' packages available for CAI/CMI and other administrative 

uti 1 ization) 



3.01 availability of, and relative performance in executing 
certain desireable functions 
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3 •01. 01 .mainframes 
' 3.01.02 peripherals 
^ ' S.01.03 operating ^ystem specifications \ 

3.01.04 interface compatibility ^ 

n > ' ' ' 

3.01..05 networking ' • 

^ / 

' 3.01.06 expansion ' : 

3.02 measures of system specification 
: 3. 02,01 clock speed. (MHz) 

3.02.02 keyboard type 

3.02.03 video display 

(resolution, character width and line length; 
line height) r^\. 

3.02.04 internal central memory 

3.02.05 internal expansion 

3.02.06 external expansion. ; 

3.02.07 internal baud rate 

3.02.08 external interface baud rate 
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To peYi'orm the bi-sectional comparison which will rglate the 
•curriculum and software sections, and the software and hardware 
sections, the modeler i,s concerned with establishing tautological 
linkages between various parts of;^each section, based upon the 
finar assessment of tl\e criteria within those sections themselves. • 
These linkages are of the usual, * logic-reasoning' specification, 
and will basically control for the existence of^(for example) a 
particular software package in the final solution set, if and 
only if: (1) the curriculum-settional presents a favorable ^ . 
criterion picture of the' instructional activities involved; (2) 
the software sectional also presents a criterion-r'elated picture 
which suggests the package is useful; and (obviously) (3) that 
such a particular software package exists. Co-relating. the ' 
software and hardware sections is identical in procedure to that 
just described for the curriculum and software sectionals; 

An additional and somewhat more complex implementTtiynnof~~tFTe 
constructs supporting bi-sectional comparisons, exists in the>j^ 
utilization of 'slack' variables. Although this tr&ipn^pt is 
beyond the scope of this particular paper, it- will be illustrated. 
for the more experienced reader. 

Recall- the algebraic relation (inequality or equality) within 
the criterion vectors as they describe their measures across all 
of the multiple alternatives, laiven that there exists some 
criterion measure appropriate for evaluating both curriculum 
sectional units and, software sectional units (that is, same 
reference arid same scaling), the measures across first the 
instructional activities can be summated and stored within a 
defined slack variable; likewise for a sum across the various 
software packages. Such a representation would exist as: 

/r 
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Act , Act Act Act Act 'Act Pkg Pkg Picg Slk Slk ' 
1.1 . 1.2 n..3 2.1 2.2- 2.3 1.1 1.2 "1.3' 12 RHS 

' ' ' ,' ' , ■ ) " 

^11 ?12 ai3 . 321 322 ^23 -1 = 0 ' 

' • Prii P12 - . P13 -1^ = P 

, , +1 •> ba, 

r +1 *1 2. bi2 

a * 

and i'n this example, assumes that the measures represent a score 
of positive benefit to be maximized -(thus the reason for the 3^ 
requirement within the algebraic inequality). 

Many other possibilities exist of course for the modeling 
of criterion references in the comparison of multiple alternatives, 
but are particular to specific situations; and therefore not 
-e as-i-l^-gen era:lixe^d7" Onc^i:hen^eBtier-m^ an d 

.constructs involved, the adaptation of the method to other 
settings is (normally?) straight-forward. 



• Illustration of Criterion Use . 

before moving on to a discussion of the various referencing, 
scaling and measurement techniques associated with data geiiieration 
techniques for the CAI-MAM framework, it may be useful to provide 
a structured example ^of how a specific type of data might be 
collected and input to the MICROPIK model . The *type' of (iata 
for^this illustration is called "synthetic", because the source ofS 
its values Is individual perception — and not a physcial ly-rigid 
measurement of some kind (like for example, weight, height or age). 

Synthetic measurement is nevertheless a most valid source of 
data for the evaluation of multiple alternatives; and therefore 
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for input to the MAA modeling framework. These measures normally 
come from one of ' two, sources, and usually must address the issue 
of 'measurement rel.iabilUy' as a more subjective, intuitive 
judgement. The usual source is the survey or opinionnaire, where 
a respondent's perceptual judgement or opinion is SQught con- 
""^erning certain issues. For example, the respondent might be 
presented with a declarative statement concerning the issue of - 
priority for microcomputer acquisition for an organization who is 
currently within -a state of 'fiscal depression. The declarative 
statement might be formulated as: - , 

. ' THE SCHOOL SHOULD ASSIGN A HIGHER Pf\IORITY TO 

. THE ACQUISITION OF MICROCOMPUTERS, -THAN TO NEW - ; 
EQUIPMENT FOR THE PHYSICAL EDUCATION CURRICULUM; ' 

and might ask the survey recipient to respond by choosing a 
position on the 6-point agreement continuum (where 1= strongly 
disagree and 6= strongly agree). As an optional procedure, the 
surveyer could list (for example) ten competing activites which 
require funding, and ask the respondent to rank-order (1,2,... ^10) 
the activities from most important to least important (of the ten 
listed). Here, a .'1^ might represent 'most important', and a '2' 
represent 'least important' (relative^to the ten presented). The ^ 
Important thing for the reader to understand (you might have 
already guessed) is, that the first option positions a high-value 
as a mor^ positive response (i.e. positive in benefit to the 
acquisition of micros); while the second option posits a low- value 
as t+ie more positive response (1st is best, etc.). The stated 
importance lies of course in the structuring of the criterion 
vector containing either the 1-6 or 1-10 values; and additionally 
in the fact that the decision-maker will discriminate between thfe 
high and low values in opposite ways depending 'upon the option | j 
chosen. - I . 
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The second sC)urce of the synthetic measure app^oximates the 
first sjy closely as to beg a differentiated description. This 
addjtfonal synthetic * type' describes the results of a prior/ 
often. physically reliable assessnrent or measure; and'which now' 
requires the' 'respondent ' s ' opinion or judgement as to whether - 
the" initial physical measure is "good^ enough", and to what extent.'- 
This form of measurement is often the perceptual results of a 
product field-test in a centr^o'lled, environmentally-related 
setting ^- where the product is put under the same conditions as 
wi ll be expected to ex^ist under .normal, user conditions upon 5ale. 
While, physical [treasures such as time, amount of'work done, type . 
of . performance, and' versati I ity or flexibility may be the physical. 
-measures7-the-user*s perception of utility and acceptability will 
als^prq^e to be very important criteria for evaluative consider- 
ation. 

The following criterion references were included in a recent 
evaluation of microcomputer icourseware; by 'the Northwest Regional 
Educational Laboratory of Portland, Oregon. (For more information, 
see the periodical "microSIFT News"*, Vol. 2, No. 1, October 1981)': 
•ftesponses'were from a panel of evaluatqrs who tested the software, 
and then offered their judgement via a 4-point agreement continuum. 
Although the reader may wish to discuss the varying degrees of 
non-specificity associated- with the 21-items, they remain still 
illustrative of ^ the means- of 'data generation, and the source of 
quantitative input to the MICROPIK model. 

The "criteria for evaluation" were separated into tv/o 
categories, content and instructional quality; and were presented 
^as follows: 

CONTENT \ 

■ \ 

■ ' \ 



206 



[01] The content is accurate;- 
[02] The content has educational value; 'and 
^[03] The content is free^ of race, 'ethnic,- sex," 
and other stereotypes, 

INSTRUCTIONAL QUALITY 

[04] The purpose of th^ package is ,wel 1-def ined; 
[05] The package achieves its defined purpose; - 
[06] Presentation of content is clear. and logical; 
[07] The level of ' difficulty is apprppriate for 

the targjet audience; 
[08] Graphics/color/sound are used for appropriate 

instructional reasons; ' . 

|09] -Use of the package ts motivational; . / 
[10] The package effectively stimulates student 

creativity^ ' ' 
[11] Feedback on student responses is effectively 

employe?^; 

[12] The learn-er controls the rate and sequence of 

presentation, and review; 
[13] Instruction i,s integrated with previous 

student experience; . 
[14] Learning is generalizable to an appropriate 

range of situations; 
[15] The user support .materials are comprehensive; 
[16] The user support materials are effective; 
[17] Information displays are effective; 
[18] Intended users can .easily and indepecidently * 

operate the program; 
[19] Teachers can easily employ the package; 
[20] The program appropriately uses relevant 

computer capabilities; and. 
[21] The program is reliable in normal use. 
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The reader can easily witness, that the 112,3,4 op^tions ft<6m the 
evaluator*s assessment could be modeled for inclusidTi witti the 
software sectional part of the^MICROPIK. A criterion constraint 
would be constructed for each of the 21 items of judgement, and 
P^^e^ 'mean-valueV responses across 'an 1 evaluators would be the 
the ejjftries for each of the vector Qomponent^; such that: ^' 



^ " j) 5^ MINIMUMt ,f or. each of the i=l,...,21 criteria; 

across each- of X-possible packages; 
where x-jj is the mean response. * 



k=l 



All criteria —-physical, synthetic or otherwise will be 
similarly moqleled, and input into the MICROPIK fj;^ework. 

ji ■ . ■ ■ • • 



Reference and Source 

Having idewiified and rdef i'ned the criterion measures v^hich 
will be utilized within the MICROPIK modeling of the CAI software 
and hardware decisioning problem, the modeler must next turn 
attention to determining 'what,* will be measured in-order to 
provide, a quantified value based. upon the construct of each of 
the variables or criteria defTped, In this context, the 'what' 
of criterion measurement is known as the criterion reference 
that is, what the modeler refers to in order to obtain' a valid 
measure of the criterion poiat identified. Then of course, the 
modeler must determine 'where' such a measure will be available* 
and/or from 'who' if other people must be involved. The 'where* 
and 'who' in this context of criteinion measurement is kno^n as 

the criterion reference source or data-point source . References 

'i 

will always involve a determination (Sf: validity of the particular 

"9 
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measure, whfle .sources wi 1 1 always necessitate an analysis of 
reliability. The readei? must recognize, that potential non- 
reliability can related to the people invoLved, the place or time 
• of the measurement, and the- procedure utilized in the measur^ent 
process — t|i*at is,, the who, where, when and vhfi\i, ' The rematnin§\^ 
interrogative adverbs of what and why relate more closely with 
the determined validity of the measured criterion point. 

References may be- categorized (loosely, 1 admit) into the 
thr-ee areas of: physical, definitional and synthetic. A physical 
reference or measure is one in which a ful ly 'acceptable tool of 

.measurement is utilized to determine the" value or weight- of the 
reference involved. „ln science, degrees of temperature, miles of 
distance, arfd knots of wind velocity are acceptable determinants 
of their associated r'Sf^nces. (temperature, distance and. wind 

•velocity). 

Definitional references are simple or comply trans- 
fo»*mations of physical measures in order to obtain a new datum to 
address a defined criterion which can ijot be measured directly'.. 
For example in the determination of school closures, a tptal of 
nine definitional crijjeri a' were designed and tested for their 
usefulness in discrlmrnating between elementary school buildings 
in order to determine their reasonableness 'for operational dis- 
continuance (Wholeberi,- 1980a). Three were-found to adequately ' 
perform this discrimination: thermal efficiency, e>iergy waste, 
and. thermal utility — by algebraically combining . a particular 
combination of such physical measures as follows: 

thermal efficiency : BTU consumption (natural gas, • 

' ' . #2 fuel oil, and electricity)',. .' 

capacity and current, enrol llnent 
of the sites; 



209 



energy waste: 



BTU consumption (natural gas, 
#2 fuel oil, and electric; ty) , ■ 
capacity and current enrol Iment 
of the sites, and the total 
dollar-expenditure for such , 
uti 1 ities; and . % ■ 

V. 

BTU consumption (natural gas, 
' #2 fuel oil, and. electricity) 

and the total dol lar-expenditur6 
- ■ „ • for such, utilities. 

For example, the definitional measure'for. energy waste resulted 
from the algebraic representation: 



thermal utility: 



I $ UTILITY I 



I $ UTILITY I 

1^ — -i 



1^ 1 

I BTU 1 

I — I I I 

|_ CAPAC J L ^^^^^ i 



\ 



We have already dealt. with synthetic measures in some detail 
in the preceding section of this paper^ Recall that synthetic/^ 
measures are normally data points of perception or. subjective 
judgement based upon personal opinion; and thus has all of the , 
reliability problems associated with subjective bias. However, ^ 
it must" be reiterated, that synthetic criterion references are 
still very much an important 'source! of data for evaluation and 
decision-making. As is the case in all evaluation, the problem 
is seldom the intent; but too often thff^ontent and process used 
. In carrying out that intent. - 
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Specific criterion references for quantifying usable MICROPIK 
data input wil T generally ^involve the use of several procedures* 
or tools. Measures related to the curriculum sectional must be ■ 
demonstrative of hot only the content and process of the various 
instructional activities, feut also the relative imporfance and . 
. degree df duplication existing between these activities in the 
prcmotiori of individual ^oncept learning. Suth, criteria as the 
.degree of achievement; amaunt of time required ;to implement the' 
particular activity, and pount of retention by student wil 1 be ' • 
directly related to the references -of performance testing via.a . 
. number of valid items or. problems, clock time, and some; form of • ^ • 
longitudinal- testing utilizing similar probl'em.'item, respectively.! / 
Criterta related more direqtly to opinion or "•percef)tual judgement - ' 
on the part of students and teachers concern^^ng the various . • 
instructional activities will be referenced by (for examplejl some 
number of statements which describe an opinion concerning some 
. aspect of the' activity, and via a survey format jgain a measure 
of 'degree of agreement' by tha respondent with respect to the 
/ particular individual items. ^ 

Gaining responses to the item (via survey techniques) 
from twj? different though related )>0pulations is i direct example 
of how synthetic measures. can be transformed in a definitional 
.composite, much. as the physical i 1 lustrations earlier . in this 
sectidn. Given responses from both studfents and teachers to an • 
identical item on two different surveys: 

"Learning how to spell using a 'spelling bee' 
■ is better than using the crass" : " ' 

K- ■ . ^ 

Obviously, high agreement on the parts of both teachers and 
students is preferred. However to control not only for degree of ^ 
agreement to the item- but also for the criterion identified as ^ 
'degree of consistency' between teacher 'and student" responses, 
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the follcwrng transformation may be utilized to provide *a 
definitional measure of consistency'! 

MINIMIZE! {teacher response mean) - (student response mean)!, 

where this foVmulation controls for betweep-gnoups consistency 
of response.^ /A similar method for t:ontrgllir>g the measure of 
consistency 'w^Chin-groups* is to utilize-^the standard deviations* 
.computed for each of the population's; .and formulated as: 



^MINIMIZEj (teacher response standard deviation) | 
and ' : 6 



MINIMl!ZE I (student response standard deviation ) I 



Measure's of degree of, achievement by students using different 
types of CA*r software -will be referenced similarly to those ^eas 
expressed above for theo'nstructional activi'ties. . Perceptual 
measures (synthetic) can also ^ be referenced via the administration 
of val^- questionnaires concerning feelings toward the exp^erie^nce 
of executing the various packages. 

■ \ ' /. . ' ' ' ■ ^ ■ 

Criterion to , permit the e\fci'luafion of the components o* the 
hardwaige sectional .will norm&lly fall within' either physical or 
synthetic, references. Such phystcal references as clock speed 
of the GPU (centraj processing unit) in mega-hertz (MHI) equ-iva- 
. lents (i.e. how many millions of cycles per second are performed), 
and of internal expansion capabilitj^ in bytes of storage equiva- 
lents; (a byte;.being a single character of input as defined by 
either an al^lia^^^ character, a numeral . (single-digi%) or a 

special characters — proviye readily understand^^ple (?) 

illustrations. More subjective judgements ^r.e also possible 
concerning the 'esthetics' of the terminal face, or the quality 



of the printer. A survey format-of the •check-list ' variety is 
a usefubtool in gaining such information. 



Through puf address ing/the issue of criterion references 
(that is, th6''what\of our needed criterion measure), we. have 
paralleled the issue of reference source, or from whe^e (whom) 
and how such information can be found (or be forthcoming) . The 
data for the curriculum sectional will come from students, 
teachers and parents — depending upon the type of criterion 
being measured. The 'process may involve the use of observation, 
a' pencil-rand-paper questionnaire, standardized achievement test, 
or. a structured interview. Information for the criterion to 
permit comparable evaluations of the software packages will- be 
measured in a ^similar fashion. Additional data for. the software 
sectional however can also, be gathered via the "dead data" . 
technique^ of reviewing brochures and records, as well as the more 
"live data" techniques of observation and survey response. 

Much of the information required to quantify the criteria of 
the hardware sectional will be found via the "dead data" search. 
Manufacturer's brochures and avai lable technical product reports 
provide such reference sources. Journal artj^cles may be also 
helpful; and so also the findings of such periodicals as the 
.'Consumer's Report'. Whatever reference and source , the modeler 
utilizes for the generation of. data points, ^the cautions coricVning 
reference validity and source, (procedureal ) reliability must be^ 
ever present in the modeler's consciousness. 

Except in more complicated MAA models related to the matching 
of instuctional activities to available CAI software and compatible 
hardware, the criterion reference for modeling, both the curriculum- 
software and software-hardware sectional will that of 'availability' 
of the appropriate software package "or hardware unit. The|^ource 
of course will always' be the manufacturer atid distributor. 
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Scaling and Measurement 

f > ■ 

.Scaling refers to the type of numeric which will represent 
the measure of the defined criterion reference; and may be one of 
four types: nominal, ordi.nal, interval -or ratio. The reader is 
referred to any standard tests and measurement, or introductory 
statistics text for operational definitions of these scaling types. 
As- a summary however, the^ types may be distinguished as follows 
('apologies in advance to those measurement specialists and/or 
statisticians among the readers); 

[Ij the nominal scale is pure categorical classification 
measure of group distinction only; the relationship 
between groups is one of difference without reference 
to either direction or extent; examples are sex (male 
V. female) and minority (minority v. non-minority); 

[22 the ordinal scale is one-step-up from the nominal type 
in that direction or order is now distinguishable for 
different responses or measures; however, the extent 
between these' directional differences is unknown, and 
provides a classic potential for interpretative error; 
examples are assigned ranks and achievement grading 
as defined by 'excellent v. good v. fair v. poor'; 

[33 the interval scale is an improvement upon the ordinal 
type in that both direction and extent (or degree) 
are now distinctive under interpretation; the intervals 
between each of the unique measurement points are equal 
throughout the .scale; examp^les are age expressed tn 
whole years, and off-spring expressed in whole units 
(formally); and 

[43 the ratio scale exhibits all qualities of the interval 
type^ and in addition allows fhfinite divisions between 
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any two points on the scale's continuum; in fact, the 
ratio scale is the only real continuum since it provides 
of the most fin.est of possible approximations available; 
7 for example speed expressed in cycles-per-second units. 

The measure (of course) is simply the numerical quantity which 
results from use of the scale in determining the value of the 
criterion from the selected criterion reference. 

The reader should note, that different measures (and often 
different scalings) can take place with respect to the same 
criterion reference — or different references with respect to the. 
same criterion identified. Measures such as these are often the 
result of a survey of opinion which attempts to gain insightful 
data concerning various issues of interest or aspects of current 
endeavor. 

The MICROPIK model will accomodate any of the scaling types 
dependent upon th? Intent of measure (indentified and defined) being 
sought. Availability of certain software and hardware units will 
often be identified as a (availability = yes) or a '0' (availa- 
bility = no); and therefore uses a nominal scaling type for final 
measurement. Presenting a group of respondents with a list of 
instructional activities concerning the satisfaction of a specific 
curricular objective, and asking them to rank-order their importance 
in promoting the learning defined by that objective, results in 
the ordinal ly-scale measure of ranks (1 = most important, 2 = next 
most important, ...). The interval type of scaling is. assumed with 
such extended continuum frameworks as the 6-point agreement contin- 
uum. And finally, the ratio scale is most usable with the more 
physical measurements ^associated with system specifications, cost 
of various software and/or hardware units versus the salaries of 
additional classroom teachers, and achievement performance measures 
on the part^of the students. ' 
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Design and Format . » 

Valid criterion definitions and references, and reliable 
sources and measures, are of course not very useful if there exists 
no technique for entry in the multiple alternatives analysis model. 
Before discussing the formatting of measured criterion data points 
in suqh a way, that the MICROPIK model will able to evaluate the 
various sectional options associated with choosing software packages 
and hardware units compatible with desired CAI/CMI objecti ves,rit 
may be best to once again review the 'guts' of the MAM framework, 
and the model's criterion-referenced, decisioning-simul.ation needs. 

Recall the design of the MAM framework as that of a matrix, 
where rows represent criterion measures across the various options 
or decisional alternatives, and columns represent the array of 
criterion measures for each of these decisional - alternatives. We 
will be concerned by the 'row point-of-view' , and address each row . 
as the criterion vector of values or simply (?), the criterion 
constraint. Since each criterion vector (i.e. row) represents .the 
values of 3 specific criterion across all alternatives, ,the reader ^ 
can easily understand how these values will be capable of validly 
evaluating the various alternatives (against themselves). And, 
since each criterion constraint can be said to therefore constrain 
the solution process (i.e. arrive at a decision), each criterion 
vector can be thought of as an 'objective' or 'goal' of the modeling 
situation, in that certain limits will be placed upon the values 
which each criterion vector can assume (as a composite summation) 
before finally deciding upon a final, most optimal solution set. 
\ 

Each criterio*nf:^Vector will be -constructecj to represent either 
a linear equality or inequality (although the inequality is often 
the more useful representation); and wi H , therefore assume the 
general form' of: 

216 



222 



^il + + ai3 + ... aij ( < , = , 1 ) 

where i-criter1on vectors have been constructed to evaluate the 
relative appropriateness of j-alternati ves, and based upon a RHS- ' 
limit to the final composite (i.e. sum), of the particular i-th 
criterion vector of the value b^. Note. that bj therefore will 
exist as an upper-bound in the inequality, a lower-bound in 
the 'y inequality, and an "identity' via an equality. . 

Thus, each a^p, for m=l,2,,l,,i criterid?i..vectors across each 
n=l,2,...,j decisional alternatives, will represent a particular, 
consistent scaling of value for each of the i-criterion vectors. 
And, since each b^, for l<=l,2,...,i RHS-values, delimits the 
^total (summed) composite which each criterion vector can assume 
dependent upon the^solution set formulated (x^ equaling either a 
•r or a- '0' depending upon the x^'s inclusion or exclusion for the 
final solution set), the particular scale utilized will determine 
'the type of objective which the particular vector is attempting to 
satisfy. ^ 

For the time being, let us set our total ' confusion aside, and 
attempt to examine each scaling type via the criterion constraint, 
framework explained (?) above. For the reminder of this particular 
discussion, we will adopt that* convention that a value of for 
the Xp decisional alternative will denote inclusion within the 
final solution set; and that .a value of *0* will represent exclusion 
of that particular x^ option from the final solution. 

The nominally- scaled criterion constraint vector can ^Iso be 
called the frequency- co^nstraint or counting- constraint vector, due 
to its use in controlling for the various frequency of a particular 
type of category within the final solution. One particular type of 
nominal control is that of assuring representative-bias -- that is, 
assuring the inclusion of certain amounts of specific types of 
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alternatives within the final solution set. To illustrate this, 
consider a MICROPIK problem which has defined five software package 
alternatives within the software sectional, and denotes the first 



two as basically 'grammerically oriented' and the remaining three 
as* Wocabular oriented' in terms of a proposed language arts CAI 
curriculum. And futher assume that constraints are required in 
order to model the following three, separate objectives: 

[1] exactly one of the grammar packa^ges must be a 
member of the final solution set; 

[2] not more than two of the vocabulary packages 
are allowed inclusion within the solution set; 

and ' ' ■ • - • 

|33 at least three software packages must construct 
the final solution set, overall. ■ 

The resulting sub-matrix of the full constraint matrix (collection 
of all criterion .vectors) would appear "as follows: 

(Objectives) x^ X2 X3 X4 X5 (RHS-value) 

I- 

[01] 11 ( = ) 1 

[02] 1 1 1 (<) 2 

[03] ^ 11 1.11 (>) 3 

(can you see that the. final solution set must contain exactly 2 
entries? and that only a total of 2 PossiDle, feasible solutions 
exist? and why additional data would be needed in order to deter- 
mine the final solutipn?). This example emphasizes the utility 
and necessity of the objective function in resolving which of the 
three potential solution sets will in fact be the most optimal set. 
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The modeling characteristics of ordinally- scaled criterion 
constraint is an extension of the nominally-scaleci constrafnt vec- 
tor. Since the terms "mean order" and "sum of order" are examples 
of the premise, "You can do any thing with numbers, meaningful or 
not.", ordinal constraints, are modeled within MICROPIK as a type 
of indicator-variable as would be found in the modeling of dummy 
variables within multivariate regression procedures. For each of 
the des.ired 'ordering points' (e.g. .ranks; or those, points ^vhich 
would' be associated with 'excel lent-good-f air-poor 'responses) , a 
separate criterion cons^traint vector must be developed 'In order to 
control the inclusion of various 'ordered' alternatives within the 
fine^l solution set. Consider the MICROPIK curriculum-sectional in 
which two sets of four instructional-activity alternatives are to 
be modeled. Each set of four alternatives has been ranked by a 
panel of experts as to" their relative .importance to the successful 
implementation of curricular goals, assigning.'!' to the most 
important, and '4' to^the least important of the four such that 
the following assignments result: ' 

■.■>'. 

OBJECTIVE I ACT-1 . ACT- 2 ACT-3 ACT-4 | 

1 I 3 1 .^\4 2 I 

2 .. I 2 ■ 1 ^3 ^ , 4- -l ,, 

and mus*t be modeled consistent with the following stated objectives: 

[Ij each objective must be satisfied; 

[23 at least two activities from each objective set 
must be members of the final solution set; 

[33 rat least two of the final solution activities 
must be of rank=l; ^ 

» 
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[4] only one activity pf rank=3 is allowed within 
the final solution; and 

[5] no activities of rank=4 are to be included 
as final solution components. 

The final modeling framework for these five objectives will include 
a maximum of seven constraints, but could be identically constructed 
with five constraints (can you see the duplication?): 



The modeling characteristics of ordinally- scaled criterion 
constraint is an extension of the nominally-scaled constraint, vec- 
tor. Since the terms "mean order" and "sum of order" are examples 
of the -premise, "You can do any thing with numbers, meaningful or 
not.", ordinal constrai|(ts are modeled within MICROPIK as a type 
of indicator-variable as would be found in the modeling of dummy 
variables within multivariate** regression procedures. For each of 
the desired 'ordering points' (e.g. ranks; or those points which 
would be associated with * excel l^^rfC-good-f air-poor'* ' responses), a 
separate criterion constraint vector must be "ifeveloped in order to 
control the inclusion of various 'ordered' alternatives within the 
final solution set. Consider the MICROPIK curriculum-sectional in 
which two sets of four instructional-activity alternatives are to 
be modeled. Each set of four alternatives has been ranked by a 
panel of experts as to their relative imp^rtance^^he successful 
implementation of curricuTar goals, assigning '1' to the most 
important, and '4'' to the least important of the four such that 
the following assignments result: 



OBJECTIVE I ACT-l ACT-2 ACT-3 ACT-4 | 
. ^1 I 3 1^4 2 1 

I . ^ 2- 1 , 3 ' ^ . j 
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and must be modeled consistent with the following stated objectives: 
[1] each objective must be satisfied; 

0 • 

iZz at least two activities from each objective set 
must be members of the final solution set; 

" [33 at least two of the final solution activities, 
must be" of rank=l; 

[43. only one activity of rank=3 is al lowed- within 
th^ final solution; and 

[53 no activities of rank=4 are to be included 
as final solution components. — 

The final modeling framework for these five objectives will include 
a maximum of s^ven constraints, but could be identically constructed 
with, five constraints (can you see the duplication?): 



(Objectives) X13 X14 X21 X22 X23 X24 ' (RHS) 



[01.13 
[01.23 
[02.13. 
[02.23'- • 
[03.0] 
[04.03 - 
[05. OJ 



1 

1' 
1 



« 1 



11 

■ 1 1 
1 

1 



(> 

(I 
{> 
(> 
(I 
(< 
{ = 



1 
1 
2 
2 
2 
1 
0 



While interval ly-scaled constraints can be modeled similarly 
to the ordinal type, careful preparation of the interval-based 



221 



J 



227 



response continuum will often yield measures closely related to 
those of the ratio- variety, and thus permit ratio-type construction. 
For this reason, the following presentation will relate to both 
occurrences of interval and ratio measurement scaling, of the 

criterion constraint vectors. 

/• . • , . 

• ■ -.■^ •. 

Unlike the previous discussion, ratio-scaled constraint entries 

are the actual criterion measure resulting from the data point on 

the criterion referenced identified. For example, in the case of 

a . physical measure related to clock time., (measured in MHz of cycles 

per second), a hardware sectional of five package alternatives 

would contain a constraint whose a^-j entries for the particular 

constraint vector would be the actual, recorded MHz quantity from 

system specifications. As an illustration, assume these five^ 

hardware package alternatives have been evaluejied on two. separate 

criteria, the first on'clock time, and the^ second on the mean 

response obtained from field-test users who responded to the item: 

-"Response time for the unit was satisfactory."- 

utilizing a 6-point agreement continuum scale'"which itself assumes., 
ratio-qualities. The tabular results of these measures were as 
fol lowed: 

CRITERION . Unit-1 Unit-2 Unit-3 Unit-4 Unit-5 ■ 

"clock'V 1.2 0.4" 1.7 : 0.9 0,1 , 

"response" ^3.5 . ^ 2.4 l.e- 3'-7 4.7 " 

and. will require modeling as follows: 

[Ij no more than two un-its must be selected as solutions; 

[23 the total sum of 'clock timeV within the final choice _ 
of units for solution must not exceed the value b^; 
and 
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[33 the total sum of 'response satisfaction' within the 
final choice, of units must be at least the value bp. 

This sub-matrix related to the hardware sectional will "be modeled 
as follows: 



ectives) 


H 


^2 


X3 


X4 


X5' 




(RHS) 


[Oil. 


1 


1 .', 


1 


■ 1 


. 1 


(<) 


2 


[023 


1.2. 


•4. ' 


1.7 ' 


.9 


.1 




be. 


[033 


:3..5 


2.4 - 


1.6 


3.7 


4.7 


(I) 





and once again illustrates the utility of the objecti.ve function 
which will be explored' in a future section. 

■ .. . ' ' ' ■ ■ r 

We will now deal more specifically with, the development of " 
the RHS-values especially needed for the st/ccessful computation of 
the exampled bj;-*and b,l used -aboveT^nd of the various methods "for 
controlling desired system impact. ./ . 
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THE CONSTRAINTS 



It was necessary, within the previous section concerning 
criterion definition, ' referencing, scaling and formatting to 
illustrate the utility and credibility of the criterion-input to^. . 
the MICROPIK model by structuring 'criterion constraint' examples. 
For the more experienced reader, it may now seem red unflant and. 
after-the-fact to conmence a formal presentation- on ^t*he id 
structure and utility surrquri^llng the .utilizatiofi of such a 
vector within a mathematical modeling framework. 



' As we have already witnessed,, the constraint vector is one 
of two* algebraic types: , either inequality or equality. This 
algebraic! format serves to. input specific criterion values of a 
defined criterion reference J^oss the avai lable /altiernatives 
into the model ; and further^ Itflizes the algebrai%r&lational 

°^>ai^ control oyeti the final alternattve^^?'^.^^v^ 

sielection (solution) procedure. In this section, we wi IT examine 
in greater detail how this control works; and how the modeler cah 
vary such control in order. to, structure a most versatile and 
flexible alternati ves"^y;alu^^^ -^.v' 

y;. .-; " . ^.a-;- V^^^^^ 0 ^ 

• • • V'y - v.- : ■ -/H-v^.^ 

Direction and Valencei -:' ./'aV;. ^-y^ 

The reader w;l T iVecall , that^^^^^ values associated- 

with eaclx parti cul.ar Criterion re^^ are intXjt to the MAM 

framework aSwCoeffi"cients-of a'-linear iTiequalfty or equality, frif^ 

the vector form: ■ - . ^ 

{. 'v,r ail ^i3 ^ij3 :^ • 

wherg the i-th criteripfr "(md^^^ objective or decision constraint) 
has distributed specif icV.^Ya;fd^^^^^^ j-alternati ves.. In full 



algeljralc 1 1 near for.mi'^ the'yectoj' of "coefficients , represerit :.■>-•, •.•■; 
series of operands of .efj:her positive or negative values 'due" to ' ' 
the. actual numerical, coefficient (e.g. a +a^-,^ versus a.- ai]^', -.for 
some k-alternative.) whose linking, operator is always the. arith-' 
met id. operation of addition. For example: ' . ; -v 

•>',.•'...'■ 

a.iixi + ai2X2 + a^axa + ... +. a^jXj; ^ J 

where each of the xj. independent ,var.i.ab>es represeht tlie various 
multiple alternatives, being, evaluated for selection or inclusi'o^^^^^^^^^ 
w^^thin the final solution set. In using arithmetic add'itibnv^^^^^^ 
Vforni^a composite of the .ai j valuG$. whose, related x-j^^ 
\yalCie ;of '1* (i.e. . inclusion ^Solution set), we assume/ 

the.cbefif>qients to be addftfVe', and thus represent ative of some 
summed eff^^ of the particular criterion reference being modeled. 

We have seen,^ that the coefficients wi H i assume different 
modeling roles dependent not only upon the reference being 
modeled, but also upon^-We type of scaling whi^cP| .was utilized for 
quantifying the criter!orf--refer^ced measurement! 'itself {rJev ^. 
nominal, ordinal, interval br^xratio). In addition, the mbdeV^r 
must also determine exactly^at effect the sum of each p^:;tfT|e 
criterion vectors will represent Jf or tie probljsm^tieTng ^ 
constructed. That is, will|a .larger sum of coefficients (viz., 
of higher value) be .seen ^linore positive (benefit?) or negative 
(unde^ir^able)". For" examp^evr.if . a' survey item whiich seeks y ^'^^^^^^^^ 
high agreement from responde?||^*on the effect of eiach of sev^^^^ 
CAI/CMI packages upon studenil^e^^^ is to be in^ut toMW -^ 
MICROPIK modeV, and the '6-poV^l|^greement Continuurfi ^^f^^ 
strong%' agree) was the r^&^j^j^ormat used for dat? collection 
— •'then the V-arious sofj^r^p will finally form the 

solution set should ^|UG.h t^^^^^ "higher'* agreement 

value than their eval^i^.y^^^^ In this example, the sum 

of those criterfJJojj, ylGfl!^ modify the solution 
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software'* at tiferii&twes",^^^^^ Jarger value, since the 

coefficients themselves . should be- of Higher * agreement V weight. 
As we will soon see, such a criterion vector constraint wiTl.be , 
called a 'maximizing' constf aiht, .:,sxnce the maximum sum* tif.V./v. . 
coefficientrvalu.es possit^Te is desj^^ : 

i-/^ ' ttf examine ^^a;'^^:^^^^^^ consider 
that triternbn constraint whose coefficierits represent the 
purported 'unit cost for each of the hardware packages being 
evaluated. Our goal of course', fjs to maximize . al t positive 
aspects of the pack^ages possible while minimizing thq^ expenditure 
required to obtain these same packages.^ In th^<s^ case,^ the .final;' 
sum .of the cost coefficients would be' pref er-abl^^^ 

■possible without compromising quality" aiid;-ut'ilM;t^^^ various'^^.%, 
alt final; .sbl4^.ti'd^^^^^^^ the 

/'smaller' the sum, the better. Sudi,:;a ^criterion vector constraint 
Will be call a 'minimizing** constraint. And as wVwill soon' see., 
-th^rd type of constraint, the'' identity' constraint, will also 

;vbe .useful when ^xact-val.ue sOms' are required from the modeling of 

^.t^ constraint. 

The 'Maximizing' Vector ConstraihtV v;^^ , # 

The vector which-seeks a higherrvaluedV^iini of the available' 
evaluative coefficients modifying "thV potent^^^^ alter- 
natives is known as/a maximizing-vgctpr/.p^^^^^^^ constraint. 
It is assumed, that the coeff icierits^.m^^ vector represent 

a desireabTev. positive influence yppn^^^^^^ decisioninjg proc^ess;- 
, and that (therefore) the higher the coefficient value of any '■ . 
particular aTternative being evaluated, the more l.fl<ely that >■ 
same aTternative will be selected as a member of the final r a': 
solution set. ■ ; .'. . ' , . 
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To assure this desi>eable. events; .the''algebraic inequality; 
relational 'greater than br-'equa] toV (V) is utilized to construct 
the criterion constraint, 'such that: ' 

^il^l + ai2X2 + aiS^S .+ •'• • + ai^xj > bf 

is. the resulting , inequality member" o'f the MAM model ing. framework, 
where the value b| is considered a. Id'wer-bound of the modeling 
constraint summation. That is, b{ is .that quantity which must. .. 
be matched or surpassed by the. summation of coefficier>ts, in . " 
order for the particular alternati.ve' solutions to be members, ■ 
of the final solution set. Until some combination of x^'s from 
the available xj-alternati ves can be found which will produce '-.' 
a sum greater than or equal to the listed b^- value, the modeling 
framework is considered not solved; and if the combination can 
not be found,, the problem setting is considered infeasible — • 
no solution is possible within the constrained decistoning setting 
as designed: • ' • 



The 'M4nimi^'1.nqr / Vector Constraint 



The vector which seeks a lower-valued sum of thfe'^avliHable 
evaluative coefficients modifying the potential sdltaioa alter- 
natives is ^known as a^minimizing-vector or min^ij^f'2at3on^^^^^ 
'it'^'U assumed, that the coefficients within the'^velrtbr^ep^^^^ 
an undesireable, negative influence upon the decisioning process; 
■and that the lower the coefficient value of any particular alter- 
native being evaluated, the more likely that same alterneitive 
will be -se.lected as a member of the final solution set (assuming 
Qf;rpourse',''that a low value correspondingly means low negative 
impact). 

\*'To minimize as much as possible the undesireable' aspects of 
this particular criterion upon the final solution, the algebraic 
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inequality relational 'less than or; equki^toV-^ C^^^^ to 
construct the criterion constraint, such that: : * . 

, ■ ^iHl + ^i2^2 + ai3^3 + ••• ^ij^j':^;!;^^ 

is the resulting inequality member of the modeling framework, ; 
wh6r-e,\the value b-j now represents an upper-bound of the modeling 
constrai nj;. sumi?ia(ti onv., That^:is, b^ is^.that the highest value ' r;,^ 
which the Vector sum is allowed to assume — and therefore allows • 
ing the sum to take on as low a value as poss.ible in its formation 
of the final solution set. As with the maximizing vector, if 
such a minimum standard can not be satisfied by the summation- ' 
across this particular vector, the pro^}„^em is declared infeasitile. 

The- -'-I-d CT t i ty ' V eetor ^ Goag traj n t 

/V> . The third and final *type of constraint which may utilized 
within any MAA modeling setting reisHhe identity-constraint. This 
vector is constructed as an algebraic equality, in the form: 

^il^l + ai2^2 + ^13^3 + ••• +^ij^j = . 

where now the specified b-j .quantity is neither (or both if you 
want to be cantankerousj^faii'V^ or lower bound on the possible 
sum of the coeff icientsV' -b^^^^ rather the exact quantity which that 
same sume must achieye^fw of the modified alternatives 

into the.rfin^ As we witness in a previous "section, 

the ideri;tTty constraint is very useful in controlling for the 
modeling of nominally-scaled criterion variables, and/or for the 
criterion vectors which represent the dummy (indicator) vectors 
of a previous 'ordinal ly-scale criterion reference. In addition, 
the identity-constraint ^s best su^^ted for controlling for those 
stringent standards which impact upon the decisioning process, 
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as. might be required by affirmative action regulations, or the 
imposition of stratif ied-group comparisons. 



System Impact Control . Via: RHS-Bounds 

Control for the construction of the final solution set is 
based upon the criterion coefficients which modify the multiple, 
alternatives being evaluated for incl'usion within thaf solution; 
and the value of the specific bound placed upon the linear in- 
equalities ^r equal it;ies being modeled within the MAM framework. 
As the criterion coefficients which modify the solution alterna- 
tives are summed for the combination of alternatives comprising 
■the solution set (where Xj^ = 1)^ this arithmetic sum is compared 
to the b^^value (RHS-bound) to assure compatibility wi=th the 
desired impact sought (i.e. <, =, or >) . When a particular set 
of alternatives can be found, such that: . ' 

.[Ij those 'maximizing' criterion vector coefficients 
modifying the members of the solution set produce 
sums which for each such criterion constraint, 
are ,greater-than-or-equal-to the established RHS- 
value(s) ; Vl . ' _ . 

* . * ■ ■ 

[2] those 'minimizing' criterion vector" coefficients 
^ ^modifying the members pf that same solution set 

produce sums which for e^ach such criterion con-/j1; 
straint, are less-than-or-equal-to the established 
RHS-value(s); and 



[33 those 'identity' criterion vector coefficients 
modifying those same solution alternative members 
produgfi/sums which exactly display the values of 
their associated RHS-bounds; 
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then a solution exists which satisfies the established constraints 
placed upon the decisioning process as identified via the various 
criterion inequalities and equalities. Such a solution is known 
as 'feasible*', and may or may not be the optimal (i.e. best) 
solution possible based 'upon the constraint matrix and RHS-v,ector. 
The determination. of optimal ity is a function of an additional 
vector of values, known as thp objective function ~ which will 
be discussed later in this section. First however, we shall 
examine in more detail this issue of controlled impact and the 
RHS-vector. 

The values of the RHS-vector are of course those bounds 
which when placed upon the sum of the coefficients of the various 
criterion constraint vectors control the selection of potential 
solution alternatives via» upper or lower bounds, or identities. 
Simply stated, an upper-bound represents the highest value which- 
is acceptable based upon the sum of 'solution* coefficients; and 
therefore most often represents a control for undesireable or 
negative effect as defined by the particular criterion vector. 
Similarly, a lower-bound represents the lowest value which is 
acceptable based upon the sum of these same 'solution* alterna- 
tives* coefficients; therefore most often represents a control 
for desireable of positive impact as defined by the particular 
criterion vector. 

Such control based upon criterion vector coefficients sums 
is a form of generalized system impact control, in that (with the 
exception of the identity) the only requirement is to meet the 
upper and lower bound restrictions^ placed upon th^ inequalities. 
Because the restrictions are based upon the composite values of 
a summatiorv, it is likely that the interactive-effects relation- 
ship between criterion values and solution alternatives will 
produce a solution set where some members may display * less than 
acceptable* criterion weights on one or more criterion references. 
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Such a circumstance should come as no surprise to the reader, as 
a particular alternative's strength on several other criterion 
vectors may outweigh its associated weakness on a single measure. 
Since the vector sum will not distinguish its individual members 
(coefficients), this particular method of control is .known as 
identifying impact to the system as a whole . ^ 

The reader n)ay also need' to be reminded at this point, that 
seldom, do decisioning situations present such simplistic settings 
as will be remediated by solutions which are clearly full-positive 
in scope — that i^, have no negative by-products or effects 
associated with them. Complex situations will always require the 
conscious knowledge of both the positive and negative impacts 
associated with the'solution(s) . The decision-maker must be 
ready to establish the required preferences in order to perform 
the necessary 'secondary choices' which will be required when 
alternative decisions present both positive as well as negative 
asj^ts to the system; and then be prepared to- acknowledge those : 
trade-offs which ajr^e .associated with the solution's rented { 
negative eff ect^^^Hi^ • V.-;^ 



Specific systein impact (as opposed'^ta;ge^^ is capable of 
being modeled within the MAM setting, via Vuch techniques as: 
selective sub-vector summations .(.controlling for marginal values 
of particular multiple alternatives), and individual single- 
independent-variable inequal ity (constraint) construction where 
j-inequa]itie^ would be required for modeling each of the j- 
alternatives for a particular criterion' reference. In most 
cases, the modeler will be able to a priori -detect if a particular 
alternative has a criterion measure which makes it undesireable 
as a solution (regardless of its other measures), and therefore 
can be excluded from the MAM procedure corr\^letely. • ' 

k Generalized system impact (which is the, preferred procedure) 
can itself be' modified or varied in order to stud/'the changes 
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in the selection of potential solution alternatives. Such a 
technique is known as the restriction or relaxation of the RHS- 
values in their constraint of the decisioning process-. 

The restriction associated with the control of the RHS-vector 
over the selection of solutional alternatives is basically a 
procedure of placing more difficult demands upon the constraint 
vectors in their formulation of a final solution set. , For the 
maximization vector, this will normally mean an increase .of the 
lower-boQnd which the final coefficient surp must meet or surpass. 
For the minimization vector, a more restrictive environment will 
mean a decrease. in the upper-bound which the coefficient sum must 
satisfy. Restriction of the RHS-values is usually executed in 
order to det.ect at'^what level of individual constraint control 
will the same solution set be constructed regardless of the 
reference of the objective function. ^ 

On the other hand, the relaxation of the individual RHS- . 
values places less demand upon the constraint summeitions as they 
measure the generalized impact of , particular solution sets to 
the syt6m as a whole. For the maximization vector, a relaxed 
state is usually associated -with a decrease in the value of the 
particular lower bound — thus making the attainance of a sum 
more easily accomplished (arfd therefore more accepting of less 
positive impact by some alternatives). Similarly, the RHS-value 
related to a minimization vector willv be increased in a state 
of relative relaxation — allowing more negative impact to be 
acceptable to the final . solution set. Relaxation of the RHS- 
vaTues is usually executed in order to generate a diverse array 
of solution alternative vectors dependent upon the respective 
iTrMuence of different objective functions. 



The Objective Function 
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In additional to the criterion row vectors." we have already 
examined as they relate to the modeling of multiple alte*^ati^s 
for the multiple alternatives analysis setting, another vector 
is necessary-to force the formation of a solution set which Is 
'optimal ' .is defined by some .a priori standard.- Unlike the 
vectors of the constraint matrix, thi^ new vector does not have 
an algebraic equivalent in the sense of an inequality, or equality. 
Called the objective function , this vector provides the basis 
for constructing a solution set vector which not only is deemed 
acceptable to the criterion vector constraints of the constraint 
•matrix, but which optimizes (maximizes or' minimizes) th'e value 
of an additional vector 'Of values or standards^ 

Thus, while the;' v^r^ious;^ 66^ and equalities 

evaluate the myVtiplev a)^^ of a feasible 

solution >(i.e.5whether a^^^ is pos^sibllB); the pbjeotive - 

function vector' chooses which of those alterriatiY.'e ;s^ s,ets 
best (most optimally) addresses a particular issue. 
the objective function* may strive toprepare a solution within 
the constraints of the problem, sq«^n>^at: the satisfaction of 
the students involved^as measured'l^bV^'^tliefr attitude is maximized ; 
or, the additional expenditures wh|c^^?Wbuld be required to pur- 
chase additional equipment is minimized.,; 

Choice of the objecti^jfeif^nctS'qn.H^^ a function of the 

overall objective(s) ojf . s^^^ problem area. 
Some modeling str eiteg^^Snifj^^^^^ 1 v a single objective 

function in the executi'oi^/,o^^ and others may 

use several in order to-;;^|^1^n^^^^^ the construction 

of the solution set. As- ji^^^i^^^ section, the 

preferred technique is t6>aj(ji^/it^^^ vectors 

serially as the objisctive \flLinGi^^^^ the differential 
impact to the formula^on p^f^^fiev^ 
each vector's ultimat4.guide;;6|%1t^^^^^^^^ 

' .- A • V .• t .'.t ''^ ' • 1 : 1 . •* ^\ «' • 



Construction of the RHS-Bounds 

The composition of the RHS-value wilh depend simultaneously 
upon the intent of the criterion constraint it modifies, and the 
type of scaling utilized in designing the criterion constraint's 
coefficients. We wi IV examine each of the types of RHS-bounds 
by its association with scaling types. This discussion .wi 1 1^ 
apply to both maximization and minimization vectors (as well as 
in. most cases, the identity vector). 

■ , \ 

Both nominally-scaled and ordinally-scaled constraints will 
normally be represented as 'frequency* or 'counting* coefficients, 
and will therefore require a RHS-value which controls for the 
total frequency associated with a particular criterion within the 
filial solution set. As was discussed previously within' the 
criterion section, potenti aV solution alternatives can often be 
criterion-addressed via measurement , scales which indicate distinct 
type or membership, rather than a .arithmetically computable value 
of both direction and degree. 

For example, consider the situation wherein the construction 
of the MICROPIK model requires crossreferencing of various soft- 
ware packages with compatible hardware units, for utilization 
within the implementation of CAI/CMI strategies. Five software 
^ip^ckages are being evaluated which present instructional activi- »: 
JiiM;|i^ to the mildly-handicapped, in the ^area of readings - : 
(iomMe^^^ Two of the packages can be implemented on one of 

systems available, while the remaining three soft- 
i^^0^j0^diges are compatible only to another hardware unit (which 
^^^Ig^^urchased if ch^osen). The problem ha3 been designed to 
■^^Iflfiife the already on-line system with the evaluation of the 
not-yet-purchased system, in order to fairly compare the attribu- 
tes of each system in relation to the potential software 

purchases. * \ 

i ' ' ' " ■ 
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For illustration, the software-hardware cross-references 
will exist as follows: 

• HARDWARE- UNITS • SOFmRE PACKAGES 

■ < A 1,2 

B 3,4,5 

gnd where an additional constraint of 'only. a single hardware 
unit* must result as the preferred solution in terms of the 
hardware sectional itself. 

Utilizing tautological constraint vectors as developed for 
the modeling of internal constraint logic subcategories for 
contingency allowance under specific inclusion (Wholeben, 1980a), 
that is: 

. "If A 'OR* B, Then C (possible)." ' 
the resulting MICROPIK framework Wbuld exist as follaws: 

I S;^ M S^S | Hii. IH I Sa Sb . RHS 



1' 1 , -1 = 0 

-2. -1 : 1 0 

1 . oi ' 1 ' ' , . '^i:^-= .-o^ 

,|' ' ■ : -3 -l' O' 

-Ji^jri-— -------- •-- ' ' * 

1 .1 = • 1 



To model the situation above, it is necessary to utilize si ack 
variables as temporary, storage locations ta denote whether any 



of the evaluated software packages were chosen by the model as 
. acceptable to the curriculum 1nstj|;uct4pnal^acti viti^ within the 
curriculum sectional, (not shown). These slack or temfwrary 
storage variables are denoted above as and Sb; and viill denote 
the selection of any of either the software 1,2 or software '3,4,5 
packages, respectively. It is acknowledged, that discussion of 
the use- of slacks (and indeed, tautologicals), is beyond the scope^ 
of this present paper. However, the reader should be somewhat 
aware of the potential for sue! manipulations of nominally-scaled 
criterion entries;'and be able .'to at i east rudimentally understand 
their utility, the third Constraint subset, which relates the 
constraint of 'one, and only one' hardware unit is to be a member 
of the solution to the hardware sectional, is a more direct and 
easily verifi,able use of the nominal-scale. 

In, consi deration of both interval and ratio measur^ement 
scales as providing the basis for the arithmetic operations of 
multiplication and division — not acceptetble to the nominal or 
ordinal measure — * the construct|pn of RHS values assumes a 
completely different perspective and rationale. Cognizant of the 
desire to control for 'general system impact' as opposed to spe- 
cific alternatives values (allowing "the model to generate inter- 
nal preference and trade-off decisions), the development of 
RHS-values will now' fol low the general ized goal : 

< to design , formulate and quantify specific b-j 
component entries of the RHS-vector for each 
modeled i-th criterion reference; such that 
. . the Ifftii vidua! b| values establish bounds 
which the algebraic inequality or equality 
relational of the criterion vector must seek 
to satisfy; and where these individual b^ values 
, . denote^ 'general system Impact' as that measure 
which is defined as the sum of the individual 
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criterion measures across the potenferJl solution 
* alternatives being evaluated^. - 

If we equate 'generaT;;ii^^^ witb^the more arithmetic t:erm • ' 
of /mean impact', thenV,the^■:^ becomes controlling the evaluation 
and final decision-nrtaktn^^:(^el^ of alternative^ for 1^ 
membersMp within the;::s^dTuta^ set) via the, structuring of scmie ' *' 
. 'mea^i value' for control Tin^^ summation of criterion va'liies 
across the various potent;Tal alternatives. In general J bri'e'might 
think of this goal as follows:. * ^ 

^il^l + ai2X2 + ai3X3.+.... + aijxj ; {<,=,>); k(MNA). 
where: ■ • . ' , " 'H^^k' 



MN;\ represet^s the mean of all a^j summed ■' ■ 
across all potential xj solutional ; 
alternatives; and 

k is some constant factor (multiplier) 
of the mean(a^j) RHS-vector entry. 



. If nn fact, our goal is to model .thy Selection of alternative : 
solution via the controTof their 'mean impact' to the system as 
a^whole, then the chosen mean term must equate the role of 'mean 
impact' to a specific numerical quantity. The value of MN;:^ does 
not satisfy this need alone, since the sum across component 
entries will often result in a quanti^ty greater than their com- 
puted mean value. However, if the modeler could identify some 
constant number which would approximate the perceived number of 
alternatives which would in turn reside" in the, sal-ution set, then 

e use of the term ktMN^) would itself approximate. the average 
impact to the system of a select k-number of solutional a-]itVna- 
tives. / ■ ' ' 
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Accept trig- a useful .metrto.dplQ.gy for developing 
^z;- numerical quantities- for descritiing mean system" impact, a new 
%; problem arises, if rin fact the MN^ value. will denote average 
impact, than a ^h'i'gh outlier' of the modeled criterion distribu- 
tion (a^j) could;,be;as large as two or three times vthe:^;Size:;^of^ 
I that, same -^dts^ mean (MN/\). Thus the use of ^'he ^ 

MNft alone'-:WQdTd>a^ bias the quantity of alternatives ch6sen for 
• the 'solution set,, since one alternative with a 2-times the mean 
'value weight for its specific criterion value entry woujd add a 
^^double-f actor to. the final criterion constraint sum of that 
particular criteripn vector across the selection solutioh.V.alter-. 
natives.^ In addition, the. MAM framework seeks to model. 'average 
••Tmpact:^ which assumes'prefejrert^ trade-offs exist^"ngV^'The, 

^computgr. -value MN/\ -1^^^ such flexi--; 

.; b i.l i ty h riherent^ wf'thi.n' the^^strij^ture of ./tl^V^ c summat i onV 

To resolvfe this dilemma, the use of the computed standard 
deviation for the specific criterion distribution is warranted. 
Identified as. SD/\, the addition or subtraction of the standard 
deviation to (or from) the mean of the-distribution— that is, 
MN/\ + SD/\ or MN/\ - SD/\— provides a readibly usable techn.iq.ue., 
for. .numerically modeling -the concept of mean system impac^t .slf^^^^^^^^^ 
. references each parti^cular criterion. It remains now.itov address 
the 5W0 situations whi^cFr warrants the use of addition or 'alterna- 
tively, the use of subtrac'irion in developing the RHS-Value. 

Addition of the SD^ term to the criterion vector computed 
MN/\ term is required for the existence of the interval or ratio 
scaled minimization [<) constraint, where the ^RHS-component 
represents an .ypper-bound; that is: 

^il^l + 312X2 +.313^3 + ... + aijXj ± k(MNA + SDa). 

Alternatively therefore, subtraction of the SD/\ term from the 
criterion vector "•computed MN/\ term is necessary when using the 



interval'or ratio scaled max1mizat:jb^^^ 



■RHS-component. represents a, lowQlp^dund; '"ji^^h^^^^^^^ 



+ 312^2 +^%3|3 t- aj^^^ I<(MNa .- SD^) . 

Recalling that the constant -k 'repre^^e^ the expected numb.er . 
■of decisional alternativesMvhid^hvwilT be finally selec^l^ as ■ , 
members of the solution set,, the'multiplication 'of either 'the : 
^ (MN/\ + SI^a) or (MNa - SDa) terms by. 'k: represents the 'mean 
impact', to be entered into the RHS-vectpr for controlling .the 
objective of that particular criterion vector Gonstrai'n't. . ■ /x^, 



Jhe' reader niay now;.- ask how such an approximation teeHnique 
could, ever be useful for modeling the algebraic felat'ionar.C=^^^^ 
of the,' identity constraint, since the po.tential/^bf|relating some ., 

.'"..specific- .:sto a compu.ted, flexible mean.; is remote. To actually.;; 

- model .Ithe id6ntitx relation.^ deci'si^rr^aker uses a matched 

pair of maxi^mizatid and ■minimization.'' cqns^^^^^ and tfjereby , 

,.y-attempts ' t.o^ doabl e-'bound the'- specif ic 'Ci^iter vector' s sum. • ■ ' ''' 
Cohstructt.qn of the RHS- values for modeling identity iippstraints 
.wi:.11..6h upon, the specffic criterion being referenced, 

but will nonetheless approximate \he -following pa " . 



.^a^4Xi'+.a^^^2' + ai3X3 + ••• + -aiW 
. . ^iixi + a|2X2 + ai3X3 +..^, . ; + aijXj > k(MNA--'- 

where the use of the. '%' factor relating ,ti the standard deviation, 
■'■term is arbitrary... In general, as the' particular criterion. • 
'reference being modeler requires more. or less convergence upon. the 
.identity of the RHS-value, the model builder will choose to^ use 

lesser, or- gr.eater fractional parts of thg^SEfA term, respectively.- ■ 



■■23-9 



ERIC 



245 



♦ Weighting Via Co-Related Vectors ^ 

We have 'discusised previously how individual criterion con- 
straints,.,could bfe more or ' less* influential upon the selection ' . • 
prpcesses of, .the MAM framewpr'k thrbugh the res_triction or relax- 
ation of the' constraint's RHS-value. Having discussed the com-' 
putation of ^the ^HS-v^lues in the. preceding topic, the reader 
should now tre able to visualize the RHS-values; and therefore 
how' the increase of a particular RHS component would restrict the 
maximization^, constraint whi le relaxing the- minimization vector's' 
process. Similarly, the decrease in a particular RHS component 
would thun reVax the maximization constraint while restricting . 
the process of the mi nimizat'ioa vector . The author cautions '^the 
reader however,, to jsmploy such \?aried and most useful techniques. 
only after attaining, initial, integer feas;ibility;|i;.e. ai^^ 
that at . least- one-^ solution exists Vas the .problefmHs . currency' 
constructed), s/iv , ■ : = - • ' ; . .: 

Another techrfique for weighting diffel^tial effects' upon ..^^v^^^ 
the final sol^ution set membership's contribution, to the/measured 
general sy St e^^^^^^^ tW'use of co-related vectors. 

TlrtS'. procedure^^^^^^ a- form Qf . str at if 1 cati bn* bfj' the aval ] abl e * 

dd'cisfbhal alternatives into groups of relative impact, based. ^• 
upon the values. for the/indi vidual criterion constraint being 
referenc^d.\'lhfi> general idec^ is to select separate alternatives 
.as being more (or less) desireable for inclusion'-within. the^ . 
final soTtltton set, based upon their individual criterion values. 
Of':co,urse, an alternative may be differentially 'desireableV due 
to/relatively positive values on one-or more criterion . references, 
while containly correspondingly negative valuSs on.'other vectors . 
Since this-^4s almost always true, the construction of the co- 
rrelated vector(s) for model "^pg weight will' often require different 
co-reljited vector(s). across different constraints fgr the same " 
aJ1ter%trve. • . • :^ ^ ' V ' . - ' 



As an illustration, consider' the problem where ten aVternar 
'tives are being evaluated for determining the final solution to • 
which- alternative instructional, activities will be fmplemented , 
to satisfy curricular objective .'0*. ••A papeT of expert teachers 
'have reviewed 'the activities, and certified each to be of v^lue 
sufficient t;o warrant their inclusion within the Jmul tiple after- 
natnyeS modeling franlework^. . This panel'-has also stated, th^t ' 
depending upon, the^ criterion.reference* involved some, alternatives 
are not only. of more positive value but also' should^ somehow be' 
weighted for greater potential enitry,,into the final solut'ion set. 
To understand their positioa^; the panel has 'identified three 

' ■ • ■ ■ ' i 

separate groups of preference high, moderate and jow) for the 
ten alternatives; and has. for two specific criterion references 
segregated these ten alteraatives into one of * the three classes ' 
of preference as follows: 

,■ ' . ' ■ • ' ■ ... t. 
CRITERION 0-1/ 0-2 0^3 0-4 0-5 . 0-6 0-7 0-8 0-9 0-10 

':-A-'^^" ^ 1 ,2 .2 3-^3 2 1 . 1 1 3^ 
\ , - . -i-B- / 2 y 3 -3: 3 1\ 1 2 2-3 1 

where a m^ti^lx-entr^^ represents > high preference, while an 
entry of '3** relates^, to, correspondingly low preference." The r 
panel also assumes that at least 6^ "ihstrurtTonariic^^^^ 
be. required, and prefer; at least 4'6f these .activities be of 
preference factor 1 or 2 on- at lea^t one of the criterion vectors, 
and. at least 2 of these 4 be fact.qi^^t 1 or 2 on -both. -.f. 

. ■ ^ . - '"'■■t^^■■^■■■ ' * • •' 

■■ ■ ■ ■ ■ • 

To ilWstrate the constraint matrix design, we wi U "Sgain 
call upon the use of slack variables as we did in the previous 
section, ut^ilizing them as temporary storage locations f>or 
within-matrix. summations. This particular example will require 
.two of t these slacks ~ one for the preference indicators associa- 
ted with criterion A, and the other associated wfth criterion B. * 

The modeling-design would then exist as. follows':. ■ 

-■ • ^ • . r) 

■ .- . 2.41 ■• ■ , . 
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' "CO-RELATE " 01 02 03 0*4 05 06 07 08 09 010 Si S2 RHS 
« ./^'^ 1- -1 ' ' . 1 1 1 1 -1 = 0 

/>B-"' € ' ^ ; 1 1 1 1 -1 = 0 

■ * "af least " , , * ' ■■• 1 1 > 4 ' 

ff , < > • ^ ^-A 

, ''^aflea^ 2" 1 '<£f' ^ 1 i > 2 

.m ■ ' ■ - . 

. i>— 

"at, ^.eHSt 6" r-. i:"^!, rM ' 1 1 1 1 > 6 

(What would liave happene^ had aa additional objective been - 
, • requii^ed which stated "at-l^ast 4 of these activities be of 

preference f actor. i or*2 dri both of the criterion constraints"?) . * 
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CHAPTER 07 
THE EXECUTION STAGE 

; Searching for Valid Solutions 
under Constrained Optimal ity ] 
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THE EXECUTION 



Now that the areas of theoretical intent, conceptual design, 
and technical development have been discussed at some length and 
with illustrations, Mt is time to explore the execution or imple- 
mentation phase of MICROPIK modeling^'-- how to gain theVequired 
results 1 of the model formulated, and what to do with those data 
' elements once collected. It is beyond the scope of this paper 
to discuss in detail the mathematical software programs which . 
facilitate the evaluation of the MAM framework. The more dis- 
cerning reader *is directed to be vigilant for an upcoming manu- 
. script publication of the author entitled, ^'Multiple Alternatives 
Analysis for Educational Evaluation and Decision-Making" — 
^scheduled for release in late 1982 or early 1983. 

This section will deal with the major four facets of the 
:fif execution'phase: cyclic optimization, the development of the 

solution tracking matrix, the creation of the various types of. 
^ solution vectors, and criterion reference weighting techniques 

based upon the various iterations of the cyclical objective fuhc- 
^ tion. The individual post-hoc analyses (statistical or otherwise) 
- which are reconmended for the results of the MAM execution, will 
' be examined in a succeeding section entitled 'Results*. 

. V ■ ,. ^- .. ■ , ■ . " 

.. Cycl ic Optimization > ^' * ^ J 

Although the multiple alternatives modeling framework requires 
only a single objective functfion for implementation of a rel'ated 
multiple alternatives analysis, the sug§est^ preferred execution 
technique is to employ a cyclical optimization procedure, wherein 
each of the criterion vectors utilized within the constrainti^i, 
matrix portion of the MAM is cycled through the model sequencially 
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..as the objective function. " In other words, given a problem of 
one hundy^ed multiple alternatives modeled across twenty criterion 
constraints, the constructed model would be executed a total of 
twenty-times, once for each of the criterion constraiints, where 
the objective function would be composed of those a^-j values 
alSQ existent within the particular i-th constraint. 

The, utility of cyclic optimization can best be witnessed in 
the statement of its goal: : . 

to generate a separate set of solution members-' ^^^'V 
based upon each individual criterion reference 
modeled within the full model,, such that -the 
. selection of these members is- based upon the 
' . • ^ same, set of criterion constraints as modeled 

via criterion vectors and RHS-values, but where ■ 
the objective function is v^aribd according to 
the- reference of the individual criterion vector 
entries. 

For the above example therefore, a total of twenty solution' sets 
would results, where the variability of membership would depend 
totally upon the utilized maximization or minimization of the 
particular criterion vector acting as the objective function for 
that execution. ^ ' ■ ' 

Each criterion vector would of course be either maximized 
or minimized as relative to it respective positive or negative 
emphasis regarding the criterion values of its vector components. 
That is, the objective will always (or at least should) be to 
generate a solution which maximizes the positive or minimizes the. 
negative characteristics of the associated criterion vector.^ 
There will moreover be occasions when selected criterion vectors 
will be both maximized and minimized (on separate runs) during 
the stage of cycVic optimization (see Wholeben and Sullivan, 1981) 
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The implication of cyclic optimal ity techniques within the 
setting of • the MICROPIK model .for selecting appropriate software 
and hardware packages in accordance to desired GAI/CMI-rel ated 
instructional applications, illustrates a '' special case for the 
application of a cycling-executable procedure. 

^ Recall the structure of the constraint matrix for the full 
MICROPIK model , composed' of criterion references for each of the 
required five sectionals: curriculum, softv/are, hardware, 
cu.rriculum-software.;^nd software-hardware. Since, each sectional, 
is' concerned with'/a s^ portion of the full constraint 

matrix,- a series^'of; zerp-sub.m^^^ empty submatrices result. 

That is, when cbncerne^d with lthe'turriculum sectional alone, the 

■ "-^' v^v- 

related row portions of the* software;. and: hardware alternatives* 
columns will be devoid of;^'arVy/4^t^^ thus, /empty \ ,::r ^v-^' 

Likewise for consideration of the ' sdT£)v>rerhaij^^^^^ 
the associated rows^of the curriculum ^alternat"iv6s*' columns will 
be empty and therefore by convention, contain zeroes for each 
of the matrix cells within that particular submatrix portion. 
Imagine this potential problem-^setting as' follows: 

CURRIC ALTER SFTWAR ALTER HRDWAR ALTER 

(-sectional-) N 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 S. 
Illllllllllll (-sectional-) 1 1 1 1 1 1 1 1 1 1 1 1 1 
IIHIIIIIIIII Illllllllllll (-sectional-) ^ 

( -—sectional-— ) Illllllllllll 

Illllllllllll (-— -sectional ) 

where, the various sectionals (or subsystems) relate data evaluation 
points either within or between decisional alternatives. 

Since some of the criterion vectors (row-wise) will contain 
major segments of zero-entries (e.g. the curriculum sectional, 
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where the software and hardware portloh.is ^6^^^^^^^ vectors 

will contain only 0''s), use of thatvv^ctiprt^;:^^^^^ func- 
tion poses the proble!?^f how the M'AM! -d^^^^^^ 

the large number of O's. For exampj^y;^^^^ reference 
is such that the objectTve of optlniiiatilg^ 

the zeros will hav*e greater influ^nfce;,t non-zero 

entries of the . curriculum sectioniaiy^dr$w^ row 

vector. On the other' hand, the bbJefiWv?^ will be 

somewhat more reliable in that the^'zero^^^^ not have .as 

great an influence as the non-zef^ i^^^^ such 
non-influence is only conjecturajl^;^ upon the 
inner-workings of the various - vectors 

The author has develope:d:'ianailifei^','t6^^^^^ seems tq 

provide not only the^raliabi^lT^^^^^^ opt'imizat'ion 

techniques, but' a\sd; assruresV^^^^ of the n'on-2er6 1 

criterion entr^ies' 'which might rbe';/usedva^^ function -i'l 

veptr^les:;;:',^^^^^ of tHfs der; thev^ol 1 ovying/t 

c1 rtiimst anoe^. rgl ated" tp' . the '^usef . p^:: fcyfcUc ^ i V^\futi6ti^gai'; . v ^ 

when the /mod^l ilig; framework: ^fi/ir^^^^ cpnstr aijit matr%y c 

numerous zerons ubtriatr i ces : 9r:;:e^ . The ob^^ecti v'lS.v pf ' , 

the objective-function is v.-:t^^^^^^^ of values whl^h tH^^^ 
MAM s'ystem can' either max depending upoVtheir> • 

measured'^rtitelrion. :Cpb^ respecti.velji^T>;v ' 

Seldom will ,the values;: 0^^^^ be numerical ty/'" •.■^ 

larger th^n .threa or fpur-;d^^^ nunibers can be * , 

expressfe'd jn ^eciijial : Un1$^ = 2.35 thousands of : 

dollars); and' smaller nunfib^^^^ agreement V 

conti nuum); can . b6 . easi^ly ; -acdc^^ -SeTdom al so wi 1 1 negat i ve ■ 

numbers, be required. : ;T^^ between thesie' 

snjal Ter . positi ve number? of ' 0 ' . has great potent 1 al ^ 

for betng^tonWuhded^^^W^^ vecitor entries is controlled 

by tilt RHS-vector ent^ , .'^ 



However, if the value of a relatively large number (e.g. the • 
value of 999999999) was substituted for the zero-entries associated 
with emptysubmatrices, and the remaining non-zero, valid entries 
left the same — the ability .to discriminate between valid non- 
zero entries and the simulated zero-entry of '999999999' is cer- ' 
tainly enhanced. The true test is of course whether such conjec- 
ture will be viable under both maximized and minimized optimality. 
Minimization holds the the least potential for confounding 
.effects, as the sum of entries within the objective function is 
attempting to attain a optimal rrtinimum value relatave to the x^j 
alternatives selected for inclusion within. the solution set. 
If in fact, the sum of all val id,.. non-zero ent was still less , 
than the simulated '999999999'''-:(7er^^^^^ chance of a-,. 

•9999999991 entry within the fjnal solution set would be extremely 
small (and maybe impossible!). 

.V ;For the, case of requirfng .the ..m of the composed,^ 

■abjecti'Ve';^^ (vector displ^aying^ impact value$),;v I 

the 'use of '999999999' wi 1 1 ^hfvftiius^^^^ as the 

use of '0* with minimization.. However by muTtiplyirvg the entire 
vector by .'-1' — that is, changing its valence structure ~ 
the new value of '-999999999' becomes as foreign to maxi^iiatidri* 
as it positive counterpart V/as to minimization. For thangmainder 
of this paper, the use of a. simulated '999999999' or '-999999999' 
Vector entry.'to. control for empty submatrices will/ be referred to , 
as '*'* and v!^*V subvectors^ ; resipecti vely. 

Solution Tracking Matrices . • 

,Each full execution of the cyclic optimization technique wi 1 1 
of course provide a solution to the problem being modeled; and 
therefore will denote which decisional alternatives were included 
as members of the solution set, ahd'which were not (i.e. excluded). 

V 0 , . , 

^ ' 248 ■ 

254.. ■ , > 



Depending upon many factors (e.g. the degree of RHS restriction 
and/or relaxation; and the criterion influence of the particular 
criterion reference utilized as objective function), it is not 
uncormion to construct a variety of solution sets as ayresult of 
the various criterion vectors utilized in cyaKc optimization. 
In some cases (in fact), a'^separate and distinct (unique) solution 
vector may resul^^ for eagh of the separate and unique criterion 
veofprs, especially under a coa(i.iVion of relaxed RHS-va-lues 
(Wholeben and Sullivan, 1981). "i ' 

The attalnance of unique solution vectors based:,upon the 
implementation of cyclic optimality is more than" just> .^n 
interesting result. Indeed, the existence of dlff e»^^ 
based upon different' objecti-Ve functi^iisjs exact l^y-rlfw^^^ 
doctpj;- ordered", when demand exists to^^5t:udy; the ef^^^^ 
upon the formutsitioh of .a particular soiujbip^^ The yead|r' sK^^^^^ . 
now be able to' understand how' three apprgaches to the never ending 
"... but, what if ..." problem can now:bej-exa^ -vf 

-^^^ as; we hal^ explored within;;the criterion sectiqlrt^ 

<:deate vyith..-the>;.ln^^ of new. cr iter iipn references withln^^;'^;:^^ 

phe modeling .fr;amewprk; and then carefully. exam iWl^^^^^^ resu'lts;;' 
of the varied solution formation^ \ The second ? jis:^ex^ in the 
preceding section ^on constraints^: discussed the^V^fl^d^^^^ 
tibn and or relaxation of RHS-values as another me'thbd W anaN 
.yz:ing the impact of: criterion bias and decisioning intervention. • 
The third technique of understanding the effect of new criterion 
references upon the solution set formation process is now -avail- 
able in the form. of 'tracking' the,^^^ solution, set vectors 
as resulting from a cyclic optimization procedure. As we will 
see moreover, the use of 'solution vector tracking* goes beyond 
the identification and recognition of criterion impact and bias;, 
and provides the main foundation for promoting such^teqhniques 
as: the integral solution composite vector , the progressive 
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criterion frequency vector , and the stepwise reformulation 
strategy for an iterative, sequential decision-making format., 

In or'der to study the imp,^ict of cyclic objective functions, 
and their effect 'upon the 'formulation v,.of a solution set vector, 
the construction of a solution tracking matrix is necessary. 
Structured as a rectangular dataset, where rows -represent the 
array of multiple., alternatives being evaluated and -columns depict 
the. individual criterion references for each of the cyclic 
objective functions — cell entries'are sl^nply either I's or 0*s 
reflecting^which alternatives were included (=1) within the final 
.solution yeq^tor based upon the maximizati^on or minimization of y 
the partl^lila^^^^ reference. /;AS"^an i 1 lustration, cpnsjde.^^^ 

%he probXeifl where;.;eight al|:ernatives-?haye been eval uated ^'across 
\-fiy^^^ (i.e. the results ' 

;0^^^^^^^^ separate execLitions of the .MAM framework) ; the simulated 
•>esults might, have existed as follows: ^ 



[,C-i C-2 C-3.g*4-e^| ISCV 



A-l| 


1 


0 


1 


0 


0 


1 


1 


A-2 |- 


0 


1 ■ 


!■ 


■ 0 


1 


1 


3 


A-3'j 


1 


0 


0 


• 1 


0 


1 


% 


A-4' 1 


1 


1 


1 


1 


1 


1 


5 


A- 5 j 


0 


0 


0 


1 


0 


1 


1 


A-6 J 


0 


0 


0 


0 


0 


1 


0 


A.7 1 


1 


0 


0 


0 


0 


1 


1 


A-8 1 


0 


0 ° 


1 


■p 


0 


1 


1 


PCFV: 


4 


2 


4 


3 


2 








ISCV the integral solution (;omposite vector, 
and represents the frequency with which 
each individual alternative 'was included 
Within .a solution set across all cyclic Tt^^ 
optimizations (i.e. the sum of the row , 'C-'^M 
vector); and >' /. > 'M-^..-^ 

PCFV =: the progressive ' criterion ' frequency vector, 
and. represents the total numberi'of solution 
^ entries whic.h comprise .the ■ solution vector 
based upon the particular criterion 
^ reference of the' objective function .(i.e. ' 

the sum of column vector). r ' 

Use .of the solution tracking matrix not qnly summarizes the 



individual>esults of th^e cyclic^ optimizations, but also provijJes 
two additional and necessary ingredients for a mone valid and: 
reliable decisioning. process. Summing the frequerrcy of solution ^ ' 
inclusion (ISCV) constructs, a 'weighting' network for the various 
potential solution alternatives. In our example, alternative '4* 
with a weight of '.S* inclusions has a disttnct .(advantage over 
alternative '2'. with a weight of '3' inclusions, or cert a In Ty . 
alternatives '1' and '3' with.weights «f '2' inclusions each. 
The ultimate goal of course is to select the final-solution set ':|- 
as that set of decisional factors which best models (or is; . . ' 
modeled by) the criterion input for the evalution process. The 
integral solution composite vector provides the necessary data ./ 

for just that evaluative need. / ' ; 

• ■ ' , ■ . • ■ ■ 

" * ■ • " ■ ,^ 

Summing the number of solution entries based upon t\)e type 
of criterion objective function, constructs a analogical ' time- • 
series mapping^ (or tracking) of the potential for further solution 
inclusion based^ upon a reiterative, stepwise solution reformula- 
tion technique.; This summation of the column' vectors (PGFV) has • 



251 



been found to .a reliable predictor of the modeling framework-'s 
potentval for generating additional decisions (solution sets) 
.based. upon tite identified RHS-valued constraints (Wholebea, 1980b) 

\, to /ti^^nsl at aforegoir]g^ paragraph into english, a prac- 
tical 11 1 Its tr at ion • might be helpful .• Consider the situation in- 
.which 5ome number of schools need to be identified for* potential 
closure according to a set of 24 agreed-upon criterion references. 
The use of* cyclic optimization (cycling each of the 24 criterion 
vectors through the MAA modeV as objective function) is utilized, 
and the technique of rejkterative, stepwise solution generation 
executed. .Simply stated, this stepwi-s.e procedure will chdose 
one and only one schoolffor closure based upon the initial con- 
struction of the IS^^V; then update those criterion vectors which 
will change value due to the closure of.* the school . selected (e.^. 
enrollment, average walking distance, energy waste); and then- 
re-execute in order to. construct a second ISCV to determine the 
second school site for potential closure. Of course, ,the question 
is how many sites will require closure 'in order to meet the 
modeled district needs (constraintsj , and how will the modeler 
.know when that limit has fcreen achieved? > 

. ■ ■ . " ■ ■, ■' 0 

: The 1980 (Wholeben) study on) school closures found, that'on 
successive iterations of the stepwise process, the values gf the 
PCFV (the progressive criterion^ frequency vector) declined in a 
consistent fashion. That is, the indi^vi/^ual sums of the column 
vectors decreased as each additional school was closed, and the 
original database progressively updated to reflect each of those 
closures; pbviously, the approach of such sums to the value of 
zero represents the inability of the school closure MAA model . 
to select additional sites for closures; and thus interprets the 
goals of the district for site closure as having been^satisf led. 

. Application of the solution tracking matrix, and its related 
components' of integral solution coiflpfosite (selection tally) and 



progressiye criterion frequenc^^ to the MIGRCIPIK model 

and; its. need to select appr software packages and' micro- 

computer hardware units .compatjible with, desireable CAI/.CMI 
ij^structional objectives —•pr^^ special case (with special 

problems) to the modeler in te»-ms ,of data interpretation; 

In the\ previous lllustrat" on, the value(sl^^ the selection 
tally vector were shown to' be. a result of summihq across each row 
of a solution tracking ma^^^ where rows represented each solu- 
tion' alter.hattve; a^ fhe criterion vectors^ used 
, as : a.^eyc1 i^c^ Qb^ectiye; function J With the MiCROPIK, model , the 
^alt^nat^y^^^ 5pl1f into. thrfee sectionals: f curriculum, soft- 
war^, and v'h rept^esentihg different though .obviously re-' 
la1^d;^decTS^ the s^fectiqn' of, appropr^^'ate CAI/CMI 
software p^ /compatible hardware deVices-to match .with 
aipara^^^^^^^^^ instruct'ional 'activities whose needs can 
be met^with^..^ same software ^nd hardware decisions. Having 
constructed the .'solution tracking matrix for the; MICROPIK problem 
the modeler in 'Vummin^ decisional l*s across each of the 
■ inherent . s^ mind, that, three subsets of 
decision-makfn^^H'a^ by the multiple alternatives 
analysis rnodefl 



il2 those /ciirr^ic^ instructional . 

, . actiykfes which wilT be* satisfied in^i^^^ 
^ .mode of instruction;' ^ ■ "V — 

[23 those'- curriculum software packages (i.e. t§!iJrse- 
ware) which will accomodate . these above selected 
o instructional activities and curricular obj^ecfryes;* 

and . ■ ■ . •"■ . : . 

l3] the particular computer hardware devices'; t'and.' 
peripherals) which will operationalize these' 



' ■ above curricular coursew'are pack-ages as "'they 
o satisfy the desired CAI/CMI i^istructional" 
' objectives. ^ ■ ■ ° ' ■ •■ .■ ' 

Aecord.ingly, "the display of the MICROPIKrsolution trackihg^(^tr.ix»i 
will be. better demonstrated as folTows: " 




i- 
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EKLC 



Ci-1 
Ci-2 
Ci.3' 



C2-I 
C2-2' 



Si-1 



S2-I 



S-j-q 



Hi 
Hz 



Hk 



I Cc.:;.| Cs....| CH....i,Cc^s...| C^H---| CCH...| CcsH..|l 

e-— : 

(1, y . ) (1,-. . . ) (i/A . ) (1, . :. ) (u . . . ) (1, . . . ) (1, ) \ 



The curricultp subsection of. the MIf ROPIK 
solution tracking matrix; where each of 
various Ci curricular objectives and the 
related Ci-r instructional activities are 
tested .for their inclusion within each of 
the solution vectors as formed by the 
cyclic Cc .... Cqsh objective functions. 



The software subsection of .£he MfCROt^IK 
solution tracking.^matrix; where each of 
various Sj curriculum and the related 
Sj-s courseware package's are. tested for 
solution vector inclusion. 



The hardware subsection of the MICROPIK; 
testing Hk hardware inclusion decisions. 



■V. 



where the appropriate criteriori'-^^^ectionals If'e represented as; 
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[Ccr =: Curriculum sectional 

SOFTWARE SECTIONAL ^" . 
CCh3 t=: HARDWARE IeCT^AL •• 
CGc<S^?S =: ^^CURRICtJLUM - StWARE SECTIONAL 
C^Sffl " SOFTOARE - HARDWARE' SECTIONAL 
[CcH] =: Cl^RICULUM - HARDWARE §feCTIONAL 
CCc5H3 ?: CURRl'f?(jLUM - software"- HARDWAI^E SECTIONAt. 



Thtf appropriate row sumnjiat^i ons across the appljed cyclic objective 
"functions' wi n now g)rasent sflection^tally vectors (ISCV) for each 
of the thrde "C", "'^'^ and "H" ('curriculum, software ^aQdj hardware ) 
subsections; agd thus deno^^ the array* of instructional activities* 
which can be s|ftisf ie^d via the. parallel inclusion^ of courseware 
and hardi(/Ve devices. It is important to note also, that with 
.the structure b'f the selection tally vector diinoting a 'range' of 
Inclusion*, the exte»tir of satisfaction i^ available for modeler* 
evaluation and ddtision-njgy<ing. 



Solution Vefc'^tors and Stepwise Reformul ationi 



The confistruction of the *ftnar solution vector, as a binary 
l^epresentation gf^the "integral solution composite vector" (ISCV, 
or se^lection" tally vector), is a lather ^straight-forward proced^e 
in most cases. The. problem ^^ual ly ^^ncoun1;|^ed will involve thi 

any 
ion 

as a decision (=1) or a nort-decision^'(=0) . ^1 

Consider the I.SCV which has resufted from a cyclic op^miza- 
tion^of a ten-alterni^ti ve, twentyrcri teri a MaT^ execution; and|nay 
be simulated as follows: > • .''"^-^ 



arbitrary decision to determine wh^t degree of Jnclusion^:for < 
parti cuJafv/alteirnative wil 1 ^ignal^^hat a/lE'ernatfve*s selectit 



C 02 07 04 09 11 18 00 15 06 .12. 3 
' 0 



0 
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t . ft 

where the fJrst alternat^ive was chosen as a solution a total of 
two-times^ the secood alternative^ a total of seven times,. and so 
forth. The sixth alternative (inclusion =18) was found to have 
the highest' selection factor,, the eighth ^ternative with the 
second highest (inclusion =15); and the seventh alternative 
never entering any'^of the cyclic gptimizations? as a probabli 
solution to the modeled problem. Based upon the range of . the' 
inclusion.^^^ shown, the final solution vector would 

be conducted by serial ly including (one-at a time) each of th^ 
solution alternatives, starting with the one with the highest 
inclusion , frequency first. Jhus, the ficnal solution vector would 
display serial development as follows: 

C 0 0 0 0 0 1*0 0 0 0 3 

c 0 0 0 0 0 1 b 1 0 0 3 ^ ^ 

- C 0 0 0 0 0 1 0 1 0 1 3 



until- 'some' ultimate criterion was satisfied (e.g. minimum new 
equipment expenditures, maximum school sites left opened, or 
minimum additional ^bu%,-stop requirements). ' . 

The necessity of such a ' test-retest ' procedure for final . 
solution vector formulation stems of course from the lack of^ 
contra* upon solution formation' by the various, criteria after the 
maximization (or minimization), of the last criterion vector during 
cyclic optimizaftion. The reader may also detect Voblems wtth 
the*notion of 'testin^-rptesting' using only an ''Ultimate' 
criterion reference ~ instead of utilizing 'all'^of the criteria 
within the original nw del. -That is, the potential exists .for 
th^ fourth .ferial configuration of' the final binary , solution vec- 
tor, , . 
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[ 0 0. 0 QJ^l 0 1 0 1 2 

to 'satisfy the ultinfate criterion (or criteria), but coterminously 

viqlate one or more of the original criterion references which 

*were utilized within the execution. of the MAM-comstructed 

problem. Other obvious . problems might arise, where the k-th 

serial configuration. of the solution vector, violates a sirigle 

> <■ 

criterion vector, halting serial solution construction ", and 
*thus preventing the further development of a more optimal solution 



(i.e. better value on the ultimatW^cfiterion). 

The final major oroblem with serial soluti^i^formulations 
exists with the existence of ' inter-dependenceV between the 
various potential solution, alternatives. For example, the closing 
y-^^-of a particular school could logical ly cause a most positive 
I effect upon a neighboring school whose own enrollment has been 
decreasing. The student transferees from the c^losed school who 
live within walking distance to the other school left open will 
obviously serve ^^to alleviate some of the vacancy problems associ- 
ated with the second site. Without taking this into consideration 
however, a serial construction of <^the final solution as described" 
above might erroneously include that sn-te as a site for immediate 
closure most embarassing to say the least. 

To control for such invalid decisions (and unreliable 
decision-making), the use of a stepwise solution generation 
system is suggested, in lieu of the serial 'system discussed. above. 
The stepwise solution technique incorporates rpany of uhe valid 
parts of the serial approach, but utilises the serial system in&, 
a more sophisticated way.**" ' I ■ 

^ Using our previous example of a simulated s^^ctidn tally 



vector, , . . ^ 

25,0. - 
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i 02 07 04 09 11 18 00 1^ 06 12 : 

the 'stepwise approach' to constructing the final solution set 
would exist as follows.- Since th/ sixth alternative (inclusion 
= 18) clearly outdistances its pbmpefr'1 tors, it would be chosed as 
the initial 'solution'; or, / V;^'' . 

. [ 0 0 0^ 0 0 1 0 0 0 0 3 

would become the first iteration resuU of the filial solution 
vector. The ojfiginal MAM-framework would then' be redesigned to 
signify the loss of alternative-six as a potential decisional 
alternative to be eva^luated across the criterion references. In 
addition, those criterion references which are related by vestige 
of- this decision (e.g. the-enrol Iment of neighboring scho6ls,:which 
would have to .absorb the student transfers) would be recalculated 
to denote -value changes (e.g. relationship of new enrol Iment to 
total capacity of the site, or the amount of vacancies). Having 
completed these recomputations, the reduced n-1 alternatives' 
model would then be re-executed, and a totally new solution 
tracking matrix constricted and selection iially vector designed. 

The result of this n-1 (or nine- alternatives) cyclic opti- 
mization might be simulated as follows: 

[ 01 05 04 08 09 — 00 14 05 10. 3" 

where alternative-six has been deleted from further x:onsideration. 
As >/as found in the aforegoing 'serial construction i. procedure^ 
the ^ghth alternative (now ='l4) becomes, carididate for inclusion 
within the final binary solution vector, or: ' , - ^ 

.E 0 0 0 0 0 4 0*1 0 0 f. , 

, A -259 ' • . ■ 



Once again, the criterion vector constraint •matrix is redesigned 
to denote the effect of choosing the eighth/^alternative as a 
^ solution^ and the ja^w' n-2 MAM-modeV n^^^^ -How- 
ver this^.third r^-execution now results in. the selection.taTly >• 



matrix,- 



L 00 03 04' 05 09^— / od — .03; 07:1, v. 



where alternative-five is the third stepwise 'candidate f^^^^ 
final solution set (instead of ^l't6rn,ativ.e-teh as. found :witK 
serial procedure). Thus the final soluti on vector; be<;o.m6si ', ■.. -v - 
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C 0 0 0 0 i-j'O 1 0 0 1, ■; ■ 



and not 



L 0 0 0 0 0 1: o'l-'d 1 1. . : ^- ' ; : r r 

The reader is caut;onecj to the:dangers'*.'of,;iiot subs'cribi.ng- to^th 
idea of a stepwise solution . strategy; and.:fgr the reasons 'W^ich 
are hopefully very apparent above. ^ V' 

The use.; of the idea of stepwi^se/ soT^^^^ 
play a primary role in thp tracking of vthe Various cyclic sol 
for ..the ■ M I CROP IK f ormu 1 at i on . ■ It 1 $ . reas onat?le^ f or |x amp le , 
expect, that only-a single type.\o> :Wand or 

will be purchased iy : ah indi vidual* 'schopV/^ %on .the re- 'vip 

suits of the MrCROPn<^e)^cuti.cJa 

of - the mpdel w^uld not .b?/4ubje^^*^-a -st^ str^t^y. ^ It 'is 
also reasor^le. to .^ssiime, that resuTts 
..(solution jlq^^ as to requi^re . 

only ''si^irt-,V^rj^f Tc^^^^ ^and selection. ;, . 

And if ^ tl^^^^(^t^ion-^^ the .partHc degree ' 

which ■ea^ft--^^ th^ curria^r- (jlj^^ 'Is" met, 'the results , 

• : ■ ■^.•^'^ ■ :W. ■> ■fe'M f , ■. ^ ' 'i 




to 



^3 



■ ■ , , ■ ^ . . -.^ i • 



■ ' will ■ exist.. as- chosen by some. arbitrar^ the tally 

. ^ 'vector's initial formulation;/ /^^^^^ ^ . V 

■ However, the stepwise technique.:can;"p 
role, in the MICROPIK. setting, jf -t^^^^^ selfiction tajly vedl 

. displays; the model 's determination that more than a> 's;1ngTe typte^-^^v::^. 
:.qf . haf dw^nre ir^^^ is required for opJtimal CAI/CWr im 

y.rij6^ritati;a^ and TR-S-SO),' It is ^uggested;tn;s|i^ 

. 1ns;tanq.es.isvt re-executed a; total of two addit^B^iy^"-^^^ 

■;■ ;;tim'e$ z^- b^^ wh^re the system Is constrained to ; choose APPC^^^ 
^..^^6n:lJr;APPLE.^^^ '.the. har.dware unit; and then where the systerp-mstW^^^^^ 
..V j- Ghqos^/;TRS-80 as ;t^ compatible^ with,; other (i^wv^^ 

■^jvf.rpm-th^ :coursev/;are and activities portions.- ; It can' be. ex^c^d;^ 
/ ; v:that: the .seleqtjiOrt t^fly^,ve^ with respect^td the::f ir * 
v", >device' solution/ wi 11 c,h;ange b:3sed upon fir excl 



■ : - TR.S-80' as a'. .d'and1jdate,: a excl.usjon of the.- ) 

' ■ APPLE. Fqf.- tha ;;^ake of .reVvev^|i^he constraints effecting eacti'^^^i^ 
. v.these yugges|ed^ nntliisiion would .exist'^r.^i^^^ 

<tar-tfc^^^^^ APPLE, and. only l\P?tE9^^^^'" 



.0 '■. 
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If 



::^^^:^;..^Af pf TRS*80, and only ,TFl|^^; ;.; 



v: 



■Jl\J.... 



"^tVereJ ; 




...■wilT'Vhoffef ull^ recall;^) the^rced ii^Tusioh 



' :of ";^the^^^^^^^^^ torthe^ cur|?,|,culiim and 

; ^bftware^^ cortstraiiM withnh the 



curr 



i£ulym-,hardj*f|re. fi^nd software-hardware, sectionals. .■ ; 

• W thesjG' tvfasfre-executions, the modeler must then - 

'•' 7 ' vif . ■■,■■ •,■.• .;■.■■■>'■■■ ■•■ 

evuluate^ hot on:ly;'-^^ extent(s) to which the new^ ' 
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two selection tally vectors solve the originally modelled Issue, 
but also the degree to which a forced, .single-device restraint 
upon the implementation. of a CAI/CMI strategy retards the actual 
satisfaction Initially desired. 

Weighting Solutions Via Tracking Vectors : 

Although the technique of . weighting particular decisions has 
been dis.cussed in the previous section (see CONSTRAINTS), it seems 
appropriate to briefly demonstrate the potential benefits of 
declaring partlcular'solutions^as more important than others. 

As demonstrated via the use of solution tracking matrices, 
a sub-matrix vector exists for each bf the results^of a cyclic 
optimization (maximization or minimization) of the individual 
criterion references.^ This sub-matrix vector,, or tracking vector, 
demonstrates which alternatives were determined both integer 
feasible and optimal based upon the values of the ful 1 constraint 
,matr1x and the cyclic objective functlorCTespecti vely. ^ That is, 
' eaeh^^&olumn vector of the solution tracklrig^ matrix shows the 
particular weight of that cyclic objective function's criterion 
re^enc^.upon the final solution (binary) constructed. As the 
va-Tue .of -9jhe objective function changed (i.e. different criterion 
reference u$ed), so often ^in most cases) does the configuration 
of the resulting solution vector.. We have found this circumstance 
to be ispecially true, where the RHS-values have been con$tructed: 
In what we have previously named the 'relaxed' state — that iS;, 
giving the, solution process ^more Mee-way* in selecting the 
best solutfcH=rromb1 nation for final Inclusion. ) 

These various tracking vector results can" be utilized to 
produce desired ;(or^-'undesi red) .weights''for the final selection 
tally '(Integral solution composite) vector. By determining the 
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factor-weights to be used in the weighting process (e.g. identi- 
fying the base-criterion objective function, and then, assigning 
.factors of related importance to the other criterion, references 
in the form of 2-times as important, 1.5-t'irnes as imgortant, etc.), 
a weighted seleciion tally vector can be formulated. 

■ ■ ■ ■ ' '■ 

Consider the following problem, where the solution tracking 
matrix,.has been formulated for the results of a f i ve-al,ternative., ' 
five-jiriteria model: 



WEIGHT: (2.0) (1.5) (1.0) (2.0) (3.0) 

j , C-1 C-2 C-3 C-4 C-5 I 

A-I: 11 0 0 • 1 

A-2: 1 : 0 0 -1 . -0 

-A-3: 'o _ 0 1 0 . 0 
A-4:. ^ 0 0.0 p 0, 

A-5: ■ 1 1 1 i 1 , 



(n/w^'d) (wgt'd)- 

NH/llcV W/ISCV 

3. 6.5 

2 -4.0 

1 1.0 

5 9.5 : 



. where tjie use of -weights does not change the direction of the 
selection tally vector, but does in fact change. the ultimate 
degree to whi'c^ti each member of the tally vector is deemed of • 
comparable value. 

Now consider a different problem, ^where the various tracking 
vectors are not so demonstrative in their selection. of potential 
solution alternatives: 
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WEIGHT: (2.0) (1.'5)-,::^^^ (3.0) (n/Wgt'd) (wgt'd) 

, [■ C-2' G72 .'. 0 C-4 C-5 I NW/ISCV W/ISCV 



A-1: 


1 ' 


■ 0, 




' 0 




1 


3.0 


A-2:. 


- "0 - 






0 


0 


■■■ 2 , ■ 


i : 2..5 


A-3: 


■ 0 ■ 


0 


: p.'" 




1 




" 5.0 




0 


0. 






■ 1 


1 


3l0 


A-5: . 


■ 0' 




■ , 0, 






■ 3 


6.5 



where weighting has provlded-aVdiscriminant pattern for further, 
evaluation, by the modeler 6f\the differences between the first 
three alternative dec'islonsv ^ ; . 

The/eader will' also note, vth^t weighting can also take .pJlace 
with the^MAM framework^ prior to'the. initial execution; of the 
niodelp^d Vtherefore be representative implicitly within', the 
lection xcmV vector. The reader is also cautioned of the 
'opportunity' for double-wei^ghting , where weights are factored 

Ynt^the various criterion- cphstrai^n^^ before execution, .and \. 

/ ■' '. ■■. . „.■,... ■ ■ , 

len utilized again as* weigftts'.^^f^^^^^^ of the solution tracking 
vectors:^as described above. - 

Application of weighting techniques to the MICROPIK setting 
has obviogs benefit for the evaluator and decision-making. By 
not weighting prior; to the construction of the initial selection 
tally' vector,, the modeler has the opportunity to witness the-; 
di.ff erehtia.l effect (if any.) weighting has upon membership within 
the solution vector(s). This is especial ly so, ^when the' analyzed 
compatibility between activities, courseware and hardware, has • 
been determi)Red . initial ly without weighting; . and then> various 
weights are applied to instructional activities anti/pr software ^ 
to note the effect upon the composite vector.' s structure. ^ 
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'However, the modeler must be ejctremely cautious of weighting ' 
strategies^ .and their impact upoh tautological requirements, via 
the curriculum-software, software-hardware and curriculum-hardware 
sectionals. . It is apparent, that indiseriminant weighting could ■ 
not apply its effieet universally a,cross the entire MICROPIK model; 
and therefore not fjroyide reliable results wifhin'the various. , 
differential selectidn tally • vectors (vfei 

the use of., 'weighted^V^sdlution^tracking vectprs is discouraged , 
except in' cases where clear 'control over tautological cross-impacts 
Is! possible. 
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. CHAPTER 08 , 
THE SOLUTION STAGE 



I Examining. Modeled-Splution Results 
■for Differential 'Cteci si on-Making ] 



15) ^ tf-' 
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.In the previous section on execution of the multiple alter- 
•natives mbclel in gener^aly and the MICROPIK^m formulation 
in particular, we h-ave been conderned with the generation of a 
^varfety of . solution vectors which would provide the . evaluator ' 
arild/p'r .decisipn-maker. usg^^ information for discriminating 
b,etween mult i pie decisional alternatives as solutions to some ; 
pre-defined problem. Specif ically, 'thiSvproblem is the conscious 
acquisition of GAI/CMr instructional software: (i.e. cpurseware) 
;arid compatible hardware (i.e. micro-computers) for satisfyihg,^^ 
array; of identified curricular objectives,, and their d^l in|^€i-^^^^ 
instructional activities.':' ' ./ 
• . ■■' ■ ' , ' .' . ; ■■' ' '■ ■ ■ ■ - At:'--'-, '" 

We Have discussed in some detail (o.r>for some of the read^^ 
too much technical' detail) ^he appl ic^tior} of criterion references 
in the form of inequalities and equal ities to the final •selection 
of a set of de<;isional . alternatives -- not pnly feasible in terms, 
of so-lving (i.e. modeling the desjred characterist-ics of) the ^ 
problem, but also optimal* in terms of .|)^;|^ 

as defined by some one Ipr more) pbjed^^p||||^ have ,• 

then witnessed how tjie individual res dlt^jH^^ 
have been, incorporated, into a solution trjacking matrix, th^ 
composite solution identified, and. differential vveightin:^^: appM^^^^ 
as desirable. . . 

. fh this section, the. concept of criterion strength wi LI be - 
be explored, as it* impacts upon the MICROPIK. problem resolution^'/ 
We will examine the related concepts of decision Validity and 
declsioning rel iabil ity, and demonstrat^e how* it qan b6 applied , 
to the MICROPIK setting. Final ly^ the use of various statistical 
procedures will be. evaluated for. their uti I ity in^ providing the 
basi^; for. some ultimate 'professional^'staitement* concerning the 



v^Mdity of the result Of the procedure 

utilized in determining threse result?. >^ J* ' V 

■*■ ' ' ' ■ ■ ■■■■ .' ' ■ ' ' • ' ' ' 

Criterion . .Strength and . D'ecisioriinq Reliabi lity / 

;„ •• Eva1uatiphy;an"d"&^^^^^ decfsion-n^afting resuTting^therewithV 
demand a high: degree *of ' accountat3i1 Ity;" visibj ^ , 
responsibi'I it)^/ , todayf5 complex issues requfre ^quaTlylcomp.lex- 
methodoj ogles to^.,^^ .bbthi content ; and' p^^^ of such issues", 
....and .to prqv.ide' an understandable'-ei^ to ^ 

/^^^^^/•^-^^^^ PDtenti aT'-decisi'ons^ and ^nieas'q}r.e^^^titti^^^^ "effect or . ' 
; fm.pi^it w ^ important, moreover , His- 1^ demand for • pro- 

.yl'ding.a^mea^ pdst^iiloc evaluating -not^ OTly th^ c>f ^ 

the simuTat'^^^ decisionsy. but alse: the influence (singularly as 
well. -^s collectively which .the icriterfon.. references lend in /, 
making the ori.girial de.cis.ibn^;. Tlie. clear lieed for * the criterion 
referehced^»decision-ma^^ the following 

f ivei objectives: 'V V ' ^ . . 

ill . to val>aate the sophisticated; declsionina'^ 
metho^dolbgies which; are so. necessary for 
addressing, cpinplex probjem^ yet so often. 
;ignoredi': discounted or fearibi ' . . , 

. ■.*.■'■;.■?/■:-■■'. • *. ■ ^ ' 

to stiidy ;Criteriori; efffect:upon 'the decisions : 
made, .and the impact which, the. system recei ves 
yta those decisidhs;^ under§tan^d . ■ • 

' - ' : •.4'if'''^e^^^^^ 

:^ r-- "wHat" made^a difference .3 n 'constructing the, 
decisions', and the varying Irnp.act re^ultlngJ'' - ; 

[3] %to..pro#de a frigh degree of vi si bi:;i1ty';.'an«d * 

therefor^\ accountability, to the public interests • 




served and affected" via tholai. decisions — 
generating a tnilTeu of trust ..within which the 
decisions, no matter how unexpected, * tan be ^ ^^s*^ 
trusted and accepted; ^ 

[4j to simulate the variable impact upon the decisions 
made by introducing additional critet^^aon influenfces 
into the model, and thereby perform a path analysis 
from solution to solution as different criteria are 
utilized to construct each decision or solution — 
satisfying the innate need of some individuals who 
must always ask, "... but, what if and 

[5.3 to permit easy and quick c^cisioning replication 
within an ever changing environment knowing 
the relationships between past successful decisions 
and the criteria u?ed to construct those solutions, ' 
!' in order to understand the potential of future 

decisions based uptin the^neW Values of more current 
criterion measures. 

Generally, the notion of -criterion strength refers to the 
'/identification of those measures which in effect constructed the 
final clecisidn or solution to the modeled problem; and further- 
more provide a 'factor' measure of ordinal value or, weight withi 
that same group of 'solution-formation* variable measures. ^ 
Specifically, criterjon strength will address three fundamental 
questions existent within all . decisioning evaluation: 

[13 which criterion references most clearly defend ]^ 
the decisions <made? ,* , , J v 



jfTlc 



l2] :to what extent are the criteria individual ly 
representative of the decisions made? 



r 
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[3] how do the most discriminating criteria within ^ 
this decision setting relate to eaolQ othet* in 
terms of importance and influence?. 

' ' • ' / 

A later part of this section will illustrate the utility of 
^discrirtiinant function(s) formulation for -answering these 
questions of criterion strength , respectively, by evaluating the 
following rudiments of discriminant analysis: ^ ^ 

[1} criteria included within the formation of 
discriminant functions that is, which 
references were 'entered' into the composition 
of the prepared functions^ 

,123 6rder-of-enlry,of each of the'variables which 
discriminate the final solution vector; and 

[33 weight (or factor strength) relationship between * 
the standardized canonical discriminant coefficients, 

■ ' ■ • ■ 

' • I . ■ ■ ' " 

Generally, the notion of decisioninq reliability refers to 

the degree of trust which is implicit to' the decision model (in 
\ » - • 

this case, the "multiple alternatives model" - MAM); implicit in 
the sense, that the decision-maker can accept the results' of such 
a criterion-referenced technology, both in terms of content 
(\nz., effect of the criterion^references within the. model ) as 
well as process (\mz ., effect of the model upon the. criterion 
references). Specifically,* decisioning rellability will addr^ess 
two fundamental questions existent within all decisioning 
evaluation: f. . 

^ [1] to what extent are the criteria collectively 
representative of the decisions made? 
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[>23;_J:o what extent can the defined matrix of 
/'V'^triterion references re-predict the original 
^ ••;.\binary (include v. exclude) solution?. 

An addtfional part of t5Ts section will illustrate the util- 
ity of dis£?f1n)inant functiqW(s) formulation for answering these 
questions of 'detisioning r^Tiabil ity , respectively, by evaluating 
the following' char act is$;tcs* of discriminant analysis: 



[1] canonical cgD.i?fijfe;1atipn coefficients which offer 
a measure o^;';^-dlationship between the ■ set' of 
discriminatill^'^'criterion references and the 'set' 
of dummy variables which are used to represent 
the solution vector; and 



[23 the frequency of mis-inclusions and/or mis- 
exclusions (or over-estimations and/or under- 
estimations) discovered when the classification 
coefficients constructed to predict a sold^on 
with the known relationships among the discrim- 
inating criterion variables, are utilized to 
re-predict the original dependent variable 
(original solution). 



Tools for. V^alidity and Reli ability Testing 



To*'t:6hstruct discriminant functions from the relationships 
:wepij^ the model just discussed above and the resulting solu- 
tions require j^the use of linear vectors and com- 
bfnat^ns of vectors (matrix). Only those ^efctor and matrix 
formi^tions most germane to this paper will be discussed below. 
The reader is invited to be patient until the scheduled publica- 
tion of^the manuscript, "Multiple Alternatives. Analysis. for 

^ V . ». 272 . / 

■ u ■ ■ . ■ . 
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Educational Evaluation- and Dedslon-f^aking" 1n late summer of. 
1982, for a detailed illustration of all vectors apd matrlce p€r- 
■ tinent' to MAM. 'I 

Solution Set Vector. In order to distinguish b,etween alt 
natives included or excluded a's members%f the final solution ^ 
the system modeled, a vector of binary-decision representations ^ 

is required^ in the form: 

\ . . . 

C 1 0 1 1 0 0 0 0 0 ... 1 3 ' ' 

4 t 

it 

where '1' means that the criterion values associated with l^k% 
particular x(j) will be computed , to measure resulting system' 
impact; and ^0' means that the underlying criterion values will 

have no impact upon the system. , - 

; / * - 9 

Selection Tally Vector. To obsgrve the eff.ect of each(j»cri- :| 
teri on reference upon consjjructtJO" of the |ystem solilftioit, a 
rnethod called cyclic optimization (Wholeben, 198fia; Wholeben and ^ 
Sullivan, 1981) is used. JJndergthis regimen, the moi^^ is^xe- 
cuted once for each unique, criterion t^ing used 1^0 constrain the 
model, where each uryque criterion is eye le<l through 'the model as 
the objective function, ^or example, duyng one^ execution ih the 
case of the school tlosure.mo^el, the intent may be be prepare a 
'solution set whereby existing capacity jof the remaining schools 
will be maximize^; in another cycl^, tfle modelwill be executed 
such that /the schools remaining" apen within the district will 
minimize the amount \)f egergy ex*pended for facility heating 
requirements. The selection tally vector is basically a fre- 
quency summation vector, compiling the number of times each 
alternative wa's chosen as part of the solution vector, acfross all 
cyclic optimizations. Such a vector will be represented as: 

'[ 3 7 0 2 0 1 ... 4-]' ■ . ^' •: "i • 
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showing that the first alternative was selected as solution a 
total of 2 times, the second alternative^' a tptal of 7 times, and. 
so forth. This vector is extreme-ly important when the MAM proce- 
dure reqtjires a step-wise decisioning procfesif such as the.^ch^l 
closure model — evaluating- a revised database 'after clqstng a 
single schooT such that the effects of closing each individual/ 
site is summarily incorporated into the next decision for deter- 
mining additional site closures, 

' Discriminant Criterion .Inclusion Vector . This vect 
represents another binary. entry <^ector of IVsi^and OAs, si 
which particular criterion references were utili^d via. 
nant functions 1:o develop the cajipnical cl'assif ication ; 
coefficients,^ and the standardized canonical discriminant ,fu 
tion coeff icients\ ^ ^ 

Discriminant Criterion Entriy Vector . This vectc^^ qontains'^*^^ 
1,2, . . . ,k, efitries, where j<^criteria . were uti 1 izedMji the. develppA^ 
ment of the discriminant functions, and the l,2',s;/f ,!< "entrj^^ 
represent their order of entry into the dis^rimthljit ffirrpulati^ 
Criterion variables not entered into "yie funct^<SfnTsOI>re^^ 
value of /O' ,' by convent i on > ' ^l&fc 

Discriminant Weighting Sgmmary V e '^Igfl L^''' ^^J^} 
nant procedures *to the binary sol utvgij^^^io^^ : y 

computation of standardized canohicci?!'^^^ 
coefficients.' These coeff icients wi fl;^^ 
entered criterion vectors Mf^'thbs'e vectl^rs^^contlin . st 
measures in lieu of ^the normal raw scores. By divi^ing^a'ch of ' . 
the standardized .canonica\ coefficients by the smaM.e'st of^.he / 
standardized canonicals,' the quotient will provide*-a factor ■ of v^'-^ 
importance for each of the criteria as relative to the other.v cri--. , 
terion entered in the discriminant for>mulation. The discriminant 
weighting summary vector is a li^near representation of these fac- 
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. tors ;(qUotien;t5),,:w^^ fninimurrf 'entry value is alwap. '^IVQQI 
. (smajTest/standardized coi^^^^ divided by itself). ^'Non-''' 

. entdr ed;.cr iter ion ;;T'o^^^^^ receive a value, of -.O.OO' by^^ • 
convent ton.. ' ' ' , . 



. • ^ Other 'tools v have been referenced in the-, proceed *g sec ;.V 

criterion constraint- matrix,; conditi<§b flimits f, ' " ' 
,' ' ..Vectqi;; ' (RHS ) i obj ecti ye f uncti on vector , and the cycl i c. optiflii za-* ■ ' 

• - • itiorijjtrack'i ng matr ix .' Other f prmul ati pns are curr entJi uftd^r . 
"V- ■ S.tuj?3^-;'?y th'^ author:- (e.g.^ ttie bptimarity weighting 'matrixif. to • "1^ 

/ i^^^^i-S^^ nev^ relationships .whi.ch may allovy greater/ac^pun-* ,^ / ".' 
^.'^i MbiJity^nd 'uS6^^^^^^ the. multiple* alteyrhaiiVes ' ' 

■• .riib;del;in'g framework; — , . . > * > ^- 

• ' ■ "' SI • ^ j>' . ■ ■■ i •.,*■«•■'.■■•,.' .1^ ' . ■ ; , .■ 'r- ' . » !» ' . * * ^ ^ ■* 

iV'/v ■'•r/'- ^■v>;..:^/■ . ^, : . ;^, , ;• ■ •■ . ., ; ■ ; 

• : : iter r^n jtrangth Vi.a. ffig Opii.itl.a1ity .' \^ei Matrix ^ V 4.*^ : ^ ^ 

,y ;explicit;/.ch.eck v^^^ (mddel) reliabi lity^by wiary.fi^ * / 

- . of;: the^;dtscrim.in^^ of se'l^^ 

^ \^'membershap);./^ahd:'t^^^ check- criterion' yalid^ty^^b^-^V 

not iipg . the^^^ istr ength' ( how much?) ^ of ; the * * 

.i.^;\;A^aripus' criterion' "^v^^ "kiscriminant analysis -i^ ^ ' 

^:.,;"\are not the sole measures.' of., p^^^^^ to the 

■ MAM deci si pn-maker. ;f urthe^ , check on val i di ty and rel iabii ity ' ' 

is ^aff orded the [Tiodejer via ;the/construc 



■ •■<> 



The optimal 1ty weighting matrix is simply a summary: of •ttie ' * > 
the preponderance of: each criterion-referen\ed variable utilized;'^ ^ ^ 
^ within the mM^^^ as measured within each subset. -of t.he^y .'^ ^ 

^spTutiart^-^;^^^^^ alternatives. The measuresVof ■ 

'•■^preponddt^^rfte Xd strength and weighting) result frpm.th^ • 

" af5plicatiofr-o#analysis of variance (^NOVA.) procedures to each of 
the modeling criteria, based upon an alternative's membership in 
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;:.;•> '>-th*4;--^^^^^^^ ablution set. .Successive ANOVA procedures can 

/v v.' 9^^^^ the criteria based upon each of the results of 



■tli6[Vc2/ol|& optp^z-ations. 

iing; an alternative as either a meniber of -the -solution 



V se*;^(th^^^ =1) or not a member (therefore, =0)V two separate . 

V;, \ dat'^il^i^r be constructed arid summarily evaluated for 

. V "bo.i^ and magnitudinal ,significance(s) of their 

v ^^^d^ii^^ differences". Since this (0,1) analysis of 



'^'-vD'^f^ procedure can; be, appTied, to each criterion, reference, 
of the cyglic optimal ity solution set results, a 
;/;'p^rti^rtrJxvfpr^^^ can be uti lized to display, and furthermore evalua- 
^v■■'^^^*iv^^^^^ the ANOVA results. This matrix is, called the 

^i..-^- opJ;4ifna^^^^^^^ matrix, where each row represents the 

; i^l^tdual cr i ter i on reference model &d wi th i n v the MAM framework*, 
)^^:^'.,, ^jwhere each column denotes the particular criterion-modeled 

y^^y^)^^ maximization or minimization based upon a single criterion 

^ ■ ■ V. ' " • . . 

B'^-'^^ ^.'^^^^^ For example, a 32-alternati ve and 24-criteria model would 

the composition of a 24 x 24 dimensional matrix with a 
'"'r''^^^^ of 1055 cells (impressed?; or beleagured?). All such 
AyV;- ■ iRatrices will always. be square matrices. 

Each of these m2 cells will be-composed of the results of 
that particular oneway analysis of variance which utilized 
solution set membership (0,1) as an independent,, variable, and , 
the. individual criterion reference (constraint vector values) as 
a dependent variable. The s'pecific statistics resulting from 
such a procedure which are of importance to our matrix, are as " 
follows: . ° : 

[1] means of both the solution and non-solution 
distribution; and their individual standard 
deviations; 



276 



0^ 



•281 



V 



[2] the statistical significance of the set. 
membership mean-diff eren^s; and 

{33 a non-parametric check (usually the use of 
chi-squared) of those criterion mean- 
differences which result from noit-ratio^ 
scaled criterion references. * 



With this summary information, the evaluator ^r decision-maker is 
able to view the frequency of statistical ly-significant differences 
betweerT the solution and non-solution sets, the direction of these 
differences and their conformance to initial constraint demands, 
the relative strength or magnitude of these differences with 
respect to degree of difference between the distribution means, 
and finally the extent to which thej'ntegral solution composite 
^vector , reflects the intent of the modeling framework ~ and thus 
the intended solution to the origi'naV problem. 



Yc|i might be thinking^ that the above procedure will operate 
correctly for ratio-scaled criterion -variable, and also provide 
.a check on well -constructed interval-scalings — but not be at 
ay useful for summarizing both the nominal and ordinal criterion 
vectors. And you. would be. most correct. Unfortunately,. nominal ^ 
data must be analyzed via contingency analysis procedures-. (or 
What most people calV.cross-tabulation or chi-squared, techniques)^ 
Obviously, mean differences and standard deviations are^not a # 
function of this analysis (or even meaninffiil ) . The evaluator ' 
will, substitute the statistical significance of -the chi-squared 
statistic, and some summary of the differences between observed 
and expected frequencies, for the usual cell . entries of the 
optimal ity weighting matrix. 

. The use of a non-parametric, numerical ly-r:ankedi, oneway 
analysis of variance procedure (e.g. Kruskal-l^|pis)''works well , ■ 
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for ordinaTly; as well as interval scalings. Some readers 'might 
think the above ruminations an adroit has^sje; but other . than the 
validity and rel iabi lity test benefits of such statistical tech- 
niques, it does support the use of interval and ratio scalings.. 
as of ten; and as completely as possible, without compromising the • 
modeling framework'. 

As you Height also have already guessed (or fearejd), the / 
applicatiorj-'of the optimal ity weighting matrix* design to the ' 
MICROPIK , setting is (oncef again) a special case. v. . 

■ • ' ^ ■ . ■ .' : ; ■ ■ ' ■,■ , 

.Because of the use of sectionals (curriculum, software, etc.) 
in /'the MAM construction, subsets of criterion references exist 
which app.ly only to specific subsets of the multiple alternatives 
being evaluated. , Therefore, some criterion vectors, will apply . ■ 
only to the evaTu^tion within the software sectional , whi le other 
criteria apply onliy'ftp the evaluation between the software and 
hardware sectionals;;' br curriculum and software sectionals. 
Application of the ANQVA procedures to the various cyclic opti- 
mizations and the resulting relationships with the full criterion 
set within the 'Constraint "matrix, should therefore (it is suggested) 
be.:directed towards each of the sectionals, rather than a system 
total approach. 



THE INTERPRETATION' 



As we approach^ the end of our -sojourn through the world of 
mathematical modeling and multiple alternatives analysis, and' 
their role in the evaluation of potential decisions concerning 
the. selection of microcomputer software and - hardware fpr CAI/CMI 
applications, there remains the need to discuss the less- ? 
technical aspects of modeling— albeit no' less important. It is 
easy to become enamored with the process of the MAM framework, 
and its role in the tllCROPIK setting, and unconsciously ignore 
the potential difficulties of the model -r both content and pro- 
cess —and their impact upon the resulting alternatives evaiuated 
and decisions selected. 

We have taken a great deal of time in exploring' first the 
conceptualization of the multiple analysis framework, ancf second 
its application within the MICROPIK structure. This was hecessary 
in order for the reader to fully understand the vast utility of 
the model as well as J end credence to the postulates presented. 
As one colleague stated some several weeks, "How can you possibly 
explain an application of your model to the CAI setting, if the 
general reader does not first understand the model ttself?". 
■This morning, I received his evaluation as to the utility of this 
paper and its satisfaction in resolving just that issue he asked 
of some weeks ago. His response was, "Oh.". But was it declara-' 
tive, interrogative or exclamatory? : / 

This last section will deal with the underlying, premises of 
the MICROPIK modeling structure, and their related positive and 
negative influences upon the decision-making require'd. We wilT 
initially examine the general . uti I ity of such a model, and the •; , 
a'filyantages to be enjoyed. In, addition, some of the more cormon , 
disadvantages and potential pitfalls of this model will also be 
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discussed; and their role tR arriving at^|erroheous conclusions. 
Finally, a totally unsolic\ited and tjiorolighly unbiased statement 
of the implications for this, technique in. future decision-making 
wi 11 be made. 



Utility of the MAA Modeling Procedure 

It nearly suffices to state, that the multiple alternatives 
analysis framework adds to the evaluation and decision-making . 
setting those components which often seem non-existent in the 
realm of educational decisioning: visibjility, responsibility, 
accountability and credibi lity. Ir> reviewing the aforegoing 
114 pages of this technical paper, what specific, references have 
been made which would allow the reader to adopt a tr^ysting atti- 
tude towards the MAM modeling procedure in gene^l, and the 
MICROPIK application specifically? : ' , 

Multiple Alternatives / The responsibi 1 ity of the evaluator 
and decision-maker is to examine feasible alternatives in resolv- 
ing a dilemma,, and then determine the most optim^il approach to 
follow. The problems associated with not identifying and defining 
all available alternatives/are well documented in situations where 
at solution to a particular problem was declared undbtainable. 
Other problems, concerned more with controlling for decision- 
maker bias and the likelihdod of pre-arranged decisions, have also 
proved the utility for adopting a multiple alternatives* orienta- 
ti on. . ' , 

. Criterion References , Accountability in evaluation^ ^and 
decision-making is inextricably linked to the data utilized in 
formulating,, analyzing and selecting the decisional alternatives 
in remediating a particular problem^tuation. The process ^in-' 
voved in identifying and defining the criteria for a required 
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. decision, the choice of datum' points for correktinq' a criterion .' 
■^reference, and the jneasurement of these points- for quantifying 

the necessary comparative values of these, defined criteria '. 

lends a visibility , to" the. evST^atoin and decision-making process 

which is fully open to public (and private) scrutiny and critique.' 

Cohsty:uents may not agree with the decisions made, but they must 
, understand the bases for these decisions, and tha Validity of . 

these underlying criterion foundation's. 

■ >>■ ■' ■ ., 

. Solution Membership . The singular, most indefensible aspect 
.of decision-making in a multiple-alternatiVes environment is the' • 
determination of si2e..and identity of the final solution set. 
^.The questions, of "how many" and "which ones" must be answered in 
a structured, scientific sense; 'and as discussed above, reflect 
both the intent and demand of the criterion references imposed ' . 
upon thiB decisioning framework. 

Interactive . Effects .' 'Seldom does^ an alternative action 

possess such qualities as to be an obvious choice for membefshiD 

within the filial solution set. More often, alternatives wi 11 ; 

.display pQSitiye characterisfics on. many .criterion ref^ences, 
■ . f ■ .•-■., .■» • 'S • 

only to denote one or two negative by-products which mi^*be un-' 

desireable to the system being moctQ;ied. The appl ication of a ° 

main-effec-ts modeling design a.llpw^«' positive attribi^es to cance^ 

the' displayed negative features; and thus !|ullifi.es^riy control > 

over such negative impact to the system ^as^ whole -rsihould. those ■ 

al tern at 1 ves be se 1 ected as • so 1 utions/ . ;.f feeracti ve /effects m.6del - 

ing'on^^the other hand -controls no't |iniy: Me ..impact o?:'p|rticu:lar?: . ■ 

subsets of alternatives upon the syS.t.em, Hut^ al,s9 individualizes: 

the. effect of each ■altWnatvve|a€ross aYl .;df its criterion measures. 

Focused Optimali1;y . The -questidns" assQt^iated with 'what i?; 
possibly'; and 'what i.s bes^' ■.■^d'dreSs all aspects o^ .decision-iipak'ing. 
The consideration of feasible^alternati ves, and the selection of 



some optimal alternative of * a1 1 , avai lable alternatives* requires 
' .the parallel choice of an overall discriminant- criterion reference. 
- THat is, once all bf our deJnands have beeYi met (ct^nstraints), the 
one, single best choice (dptimal) must be found based upon some 
predefined point of reference (objectfve function). . Several such 
points of reference (cyclic optimization) allow the ,decisioner to 
- examine tha,impact of potential alt^rn'atives upon the^environment 
being' modeled. » \^ ^ 

Tr,ade^Offs and Preferenc'es . Since alternative solutions wi 1 1^ 
' . often display both posvtive and negative' attributes regarding thefr 
probab'te impact(s) to the system,, decision-making must be-^iable- to 
rel iably monitor both the di recti on • and extent qf effects: to the , 
I system be remediated.; Whi le many side-eff tfcts 'may *be^ 

the quality .of each alternative's positive characterfstics must^' V 
TDe alTpwed to model the. desireab'Ve'benefttsVof that alternative, 
Simultaneously, positive and negative char actenis'tics must be 
allowed to co-exist and therefore tje meaiiSreible, in 'order to V 
truly model the real-worltd situation. 

Stepwise Solution Formation . : Since some aspects of;any 
decision impacts upon other .decisions which may be fprthcomilig, 
preparations must be mcide to xontroT for the effect^jof s.uQh ' 
. preceding ._decisions upon potential succeeding decisions -which may . 
. be necessary' to completely^ satisfy stated constraint requi elements. V 
'With a. criterion-referenced dataset 'as the basis for compairative 
, , eval^tion among alternatives, the selection of a single decision 
. . will obviously, effect the criteVion values in some way ' (assuming 
>/. './of caurse, that the decision does in. fact provide some degree of ; 

ifemediation to the system be modeled). In order to evalliate'^e 
'V. . ' /remainder* of the system prob1em*,vthis dataset.must be updated ^ 
/ \'to reflect the degree of solution already imposed, by'the choice 
- of th& previous decision (alternative selected). -.Subsequent / 
analyses will then be able' to provide a valid and reliable 'nextV 
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sonutfpn "^0 "Jmpact ,u of ^problem "^remaining' .< - The 

final entr^ set is - reached, when a. subsequent . 

analysis fafls to detect';; a; new member; and the Tast dafaset ;updat 
refld^S','. system, vent erf components as desired, 

■ Simulation (Before) ^and Interrogation (Af-ter)V The fjnal* 
measure of ' uti 1 Ity f-or the MAM > formul ation Ti es ■ in . its- ^bil ity to 
^provide; botft nnductive and. deductive-reasbning^mectiahics- td'the ^ 

system" evaluator decisionrfnaker. Basied upon a carefully " 

' ■ : • ' ' '• ■ . ■ ■ . ■ '■ . 

derAved'set of ^ciliterion-referenc^ which ^re* deemed' represent ati 

'"of both, th^^stgfp..^ and, alternatives which possess, 

■ -a varyi ng i^^re^';^^ PP^^^ ' '^^s o 1 Ve an i tient i f i ed prob 1 em' . 

within':'thcj:^*^sy^t^^ .the mul'tiple .alternatives^ model is Ible to 

simulate. |lte^ problem settling, • and :thus derive (viz. , indu,ce) the 

•niecessary ••^Jijtlbns- which^^^ the d^mand^s and needs of *the 



system: i^Qr^eov^r In the case where^ecisions have already been • 
presumedvbSs^d ' upon: some;' set of^ criterion measures., th'e MAM 
f ramework ;is^^ible :tb-^^^^ setting, an'd'thus- 

derive vfi^^,ded'uce) the;. de'mands. : and ' needs of the system which 
refj-ect the^pHo^^^^^ in the\event of 'a set 

of decjs.iorii|^ an Identified criterion- 

referenced database, reasonable crfteria can be pos|;ulated and >^ 
subsequently me^red against the proposed solution set. 



Advantages :an(^ 1j ^ 'of MICROPIK ^ t " - ■ 

*. The' need t*-test the.cQntent' of decisions for validity, and 
t+ie 'process utilized in arrived at this' gonteht for reliabiTity, 
.suggests" the feather superfluous^aissertion that- any technique :for 
m^aking decisions has its problems* in 'addition/to its- laudable 
• benefit$v- ;Tbe MICROPIK modeling- forgiul atfon is ("regret ably) no 
exeeption'-td this existential assertion. 

■ ■* • ■ ■ • . ' ■ ' .»'*••- ■ • ' ' ^ 
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Recall the main goals of the iMICROPIK structure: to provide" 
a criterjpn-referenced, multiple-alternatives decisibning model 
for evaluating CAI/CMI software and ipicro-computer hardware for 

, 1 ts comoatibility with desired curricular-objectives and instruc- 
tional activities; That the modeling framework as exposed within' 

^the preceding paper actually accomplishes this task, posits the 
main advantage of the model over , any other decistoning tool known 
to this author. 'As^ stated within. the preceding, section goncerning 

"the utility of the 'mam procedure iJ^^^^eneral , specific advantages . 
are assignable to the MICROPIK frafSSewlirk in t^rms of its ability 



ll] provide an .evaluation framework for tf>6 "".^velop^ ' 
njent of a set of decisions (s^olution set] ifntpm 

a larger set of potential, multipTe alternates; 

■■ * - ■ ' ■■ . ■ ^ 

[23 utilize a criterion-referenced datas^t as 'the 

basis for comparing the direction and degree of*/ 
positive and negative attributes associated with 
^. each of the potential, mu.ltiple alternatives; 

[3] control for the interactive effects between the 
measured criterion values., the various groupings, 
(combinations and permutations) of -the multiple 

I alternatives, and their resulting impact upon 
the* system as a whale ~ and thus determine the 
members of the solution set in terms of "how * 
many'*,_ and "which ones"; 

[4] investigate the eff-£Ct of varying the optimal ity 
design for each "^separate execution (solution set 
formation) and thus examine sequentially the 
biasing factors associated with each criterion 
vector; ^ , . . • 
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iS] prepare a'deitabase revlsTon strategy for imple- 
menting a stepwise-vprocedure in deve^pijig the 
final solution set; and ^ 



[6] • simulate t^he impact upon the'system as a whole 
*of potential decisions for remediation of the 
defined .problem, as well as interrogate the 
relationship of past 'and/or current decisions 
to thetagreed-upon criterion references sup- 
porting those decisions. 



.However, the implementation of the MICROPIK-mbdel also has 
a number of disadvantages -associated with its utilization — as 
therefore does the MAA model in gen'eral. These disadvantages can 
be encapsulated within three general headings: model -related, 
user-relatedy and equipment-related. 

Model-|^elated disadvantages are probably obvious to the 
reader at: this point. The development of all possible, or feasible, 
solution alternatives, the definition of all sufficient and' 
necessary criteria, the scaling and measurement for each coeffi-/ 
cient entry to the criterion contraint vectors, and thp concep- 
tualization and computation -of the appropriate RHS-v^lues -- are 
enough to divert evaluator Interest to other less-sophistioated. 
evaluation techniques; and have been known to drive even -the .most 
adroit . educational administrator to fits of manic depression. 

User-related dils'advantages are foreshadowed by , the initial 
use tDf the terms ^mathematical modeling' and * simultaneous lifiear 
inequalities*, and^ the* tendency ,pn the part 'of the administrator 
to request inmediate psychotherapy. These exist sufficient his- 
torical references to past evaluators who have utilized quantifi- 
able evaluation techniques to maske the. real missions of their 
endeavors, or to provide post hoc support to a priori decisions 
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devojci of a valid criterion-referenced framework. The use -^Of any 
new Wrmino'logy .Is greeted (with the criticism of "jargon"; the 
use Qf ^mathematical techniques with the criticism of /"... not ' 
everything can be quantified ... and. therefore,- notl>ing should; ■ ' 
be ..."rand the use of sophisticated tiecisioning strategies and 
(as we will soon see) electronic computers with thfe/ criticism of 
"... too hard to understand ...^^too technical for consumption 'by 
the general public ... and therefore^ notVseful ./.". ^ 

Equipment-relatgd di^advantaq^^s are in -reality the true in- 
surmountable barriers tO' the accept%ce and subsequent use of any 
MAM design. Although -the author h^s'^on occasion (but infrequent- 
ly) solved MAM problems "by han'd" ~*^this is not the preferred 
technique. Therefore, the- use of .computers is the modeler's 
salvation. But to utilize these computers, specif i,c,. software 
packages must themselves be availa.ble (or written) to correspond 
with the required mathematical )3rogramming algorithms* needed for 
MAM solution. While such packages are available (e.g. IPMIXD, 
MPOS, EZLP, LINDO), they are nof a usual software component on ' 
most computerized hardware mainframes. And unfortunately, most 
evaluators have, not been instructed in their use, -let alone theV 
existence .and utility. . , , • ' 

Fiiaally, the MICROPIK formularization 'of the general MAM 
model i§ (unfortiinately 'though not apologetical ly) a complex 
variatioa of the multiple-^alternatives, integer programming sys- 
tem. The use of alternative sectionals, and separate criteria 
j:o relate various sectionals for cross-evaluation, adds to the 
potential ci5nfusion and conflict on the, part of the us.er and the 
public whose needs the modeler Is attempting to satisfy. 



/ 

/ 



Currently, the evaluation of curriculum, courseware and / 
hardware for CAI/CMI implementation proceedsMn an offtn undefined 
manner t- har.^lware is purchased; the.existing compatible software 



v286 



Of' ■ 



291 



is: examined; and curricular objectives or instructional activities 
redesigned ta fit tbe available courseware perogatives. In effect, 
the classroom teacher ist 'locked-in* to a hardware device, which 
in'turns narrows the choice :of software^-.and thus ultimately de- 
fines the satisfaction of particular objectives. More and more, 
school districts, are examining software fiirst, then. the hardware 
•devices for compatibility, and so forth. iWhiTe.the newer trends 
in evaluating CAJ/CMI are producing more satisfying results, the 
ability to control for all multiple alterrjative instructional 
activities while satisfying- (to some^egreie) all curricular ob- 
jectives, and relate these to the avaWablje courseware and hard- 
ware—has not, been pos'sible (until MfCROPIK, obviously). ' 

Major PitfaNs and Erroneous Conclusions 

Within the consideration of advantages versus disadvantages, 
we- a priori assumed. a successful design, construction 'and execu- 
tion of the MICRQPIK model. Now however, some time must be ex- 
pended in discussing the potential problems associated with the 
inappropriate design, invalid construction and/or unreliable ex-, 
ecutlon of the modeling framework. _ ^ ^ 

As has been reiterated throughout th'is paper, inappropriate, 
design is usually associated' with the exclusion of some alterna- 
tives (Wr whatever ceason) from the modeling framework. .For 
example, ^he aBser^e of various instructional activities and th^ir 
relationships to potential courseware availability w'ill automati- 
cally preclude the model's potential in satisfying their needs. . 
Likewise, the absence of a particular criteripri from considerjition 
will preclude the model's ability to control fo^' that criterion's 
.impact upon the -system which may be poi^itive or negative^ \ and/' 
maybe even disasterou*s.^^ . * * " ' 



The ap^pearance of invailid constructioh i|as a major; pl.tfall 
often ta'kes .the form' .of problems associ.ated:;w1th the sca>ing and-i^ 
measurement of the .criterion coefficients (vector xqmponents) ; 
and is thus a s'econdary prptilea stemming from inappropriate . . 
criterion referencing*. Problems will 'also arise based upon the^ 
criteriori's'measure'and iis , uti lity in -describing system i'mpa,c\ - 
based upon^ a-row-vectoV summation,- • ^ . . ^ 

Unneliable . execution ,is a frequent problem associategl 'wi.th 
the coilstruction of the RflS-vector-, agd the' complex f-estriqtion 
versus relaxation effect these values have upon the summations 
of theMndividual criterion constraint vectors. The useV;.of the 
^cyclic optimisation strategy also .provides difficulty -for^^lie^ 
•ma'inteaance of ' rel iabi jfty; indiscriminant maxim^zati^^^ 
imizarion) ' can introduce conflicting. demands to the system, arid 
produce solution sejTs irr direct opposition to one another.; In 
addition, compilation of the various cyclic solution vectors 
into ci final se^lecti^ tally veqtor (though vatid) can also pro- 
'vide a^new source of unreliability to the final determin-ation of 
the actual binary: solution vector. • 

In;general however, once all of the procedure^J, techn-ique- 

oriented, and sequentially-defined prerequisites 'have been; met, 

the major problems associated with the MIGROPIK modeling 

\ • 1 ^ . 

situation remain: first , its interpretatiopyfor decision-making; 

and second , its incorporation into practice, ' 

^ * ' ■ • . . ■ 

• < >• ■ • . ■ . ^ 

The interpretation of MICROPIK results must include a firm 
understanding of the. MAM process, and its ev-kl^ation structure. 
ThisMs the reason for expending^ the time and'energy in the . 
current development of this research paper. Individuals who 
accept the premises ^ upon which the MAM technique. is built, and 
the postulates of njultiple al tern-atives evaluation and criterion- 
referenced -control — must also accept the- notion of trade-off 

\ - , • . ' ■ 

•. . ^ . ■ ■ • * ' 28C ■ • ■ ■ ' , 



and preference structure, and optimaVityT A common pi^oblerh has . 
•^freqyently been^the erroneous conclusion,, that the model's 
decision concerning solution membership is d.evoid'of any negative 
3 impact.- Other misinterpretations surround the. idea>, of 'what was 
the problem ^s defined (?)', and therefore d(jes the' solution 
vtruly' solve the' p^roblem", or .merely cop^ vfith the actual pr^gblem's 
: negative impact. -^For example', closing^schools do6s not so-lve the 
•problem of declining enrollment, but does permit a rational and 
accountable means of coping with Its effects. Succe-ssful execu- 
.tion of the MICRO'PJK mocTel wi ll prov.ide th^ best/.f.it of course- ■ 
ware and hardware with , desired activities but imay not be- able 
.to qeet alT of the desired needs. / limitation; .of a 'sing]e • 
hardware device, and a p^rti culaiy preponderance of oQ^unseware on 
a particular hardware unit,^ayf^uire the sacrifice of a single 
discipline's CAI requiremerfts.'due to non-compatible software on 
the preponderant device chosen. ' ' ,' 



The ^incorporation of^MICROPIK results into practice must a ' 
never be the result of solely fol lowing the binaryy ndicators of' 
the final solution^* set vector. The modeler must Yre/:al 1 , that the 
membership of the solution vector resulted from -a nfathematical 
analysis of a number of . criterion-oriented inequalities/ which 
themselves wece products of definition,- V^eferencing, scaling and 
measurement — and therefore all of the^ problems associated 
therein. The decision-maker must look upon .the MICROPIK resutts . 
as structured, controlled ;suggestions"; and in many cases, just 
further "input" to the decisloning process which always re^ts 
in final form with a fles-h and blood person. Contrary to public, 
wide-spread predicti oris . of doom, technology will never replace 
the human decision-maker — Although the potential is there to 
make that decision-maker more valid, reliable and hopest. 



Implications for Future Applicati on 
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In clbsing this most laborious but very sat i'bfying. project,, 
the forthcoming criticism ^om individuals who believe nothing 
(or at least, choose not t^) unless if ,is accompanied with reams 
of data, print-outs, must be addresse.d, and their concerns fully 
acknowledged.. . : . . " 

' A full p'.iloting or field-test of the MICROPIKVmodel , -and . 
its resulting effectiveness and efficiency in 5electYiaa^micr-)f 
computer "hardware, compatible instructioilal software (i.e. 
• courseware), and related CAI/CMI curricular objecti ves^/and i 
instructional ^activities has as of the^date of this/^per not 
been accomplished. In' fact, the author is currently developing 
^ a greater, diversification of criterion needs and references for 
input to the model. Field-testing of 'the tnodel is currently 
scheduled for the autumn ofsl982i and is expected to involve a 
large number of fchool distrra:s Mn order to obtain sufficient 
;-fr e q u e n c>-e s- of^o b s eryati o n-to-arf ord-th e-n e ce ss ary - cr oss-compar-. 
isons between model types, -and 'Supported, software packages. It 
is also the intent of this author to involve each of the major 
^hardware and s-oftware distVibutors (as much as possible) in the 
design, develftpnlent, construction and final implementation of ^ 
the MICROPIK mocfel. Obviously, such coordination requires a 
great deaj of lead-tfme; and much to my chagrin, can not be 
modeled in/a multiple-altprnati ves setting (or can it?). 

Another obviously major portion of the intended pi loting of 
the MICROPIK formulation will depend upon the ability of'school 
districts to define their desired CAI/CMI needs; and then relate 
these needs to specifically definable and measureable instructional 
activities. States such as -^^ASHINGTON which have begun^ concerted 
efforts to direct each school district to develop "student learn- 
ing objectives" CSLOs) for each disciplinary or curriculum area, 
will provide greater facilitation in the final derivation of CAI ^ 
and CMI curricular objectives and in-structional activities. And 
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of course, only tho|e school' districts which have the necessary 
microcomputer hardware and coursewiire will be included in the 
project if they so desire. Large purchases ;for data processing, " 
technology is not a priority item for districts jvvho are currently 
forced to RIF classroom teachers due ^to budgeting problems, x 

The interested readen. is invited to contact the author, and 
begin comnuni cations which might provide a basis, for. cooperative 
ventures in sS^isfying the uncoming requirements " for ' a ful.l-s^cale 
field research^, Others -are invited to- stay tuned to^Tbi^ther . 
developmehts in the^MICRdPIK process, and its impact upoS the ' ^ 
general evaluation and ■"decision-making structure currently found 
in 'most 'school dis'tricts ... same BYTE .''time ... same BYTE channel, 
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CHAPTER 09 

J' • 
THE UNDERSTA^NDING SfAGE 

L Illustrating the Stepwise Implement atton 
f a Fi-eld-Based Quantitative Application ] 
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THE FIELD APPLICATION. 

■ : ■ ■ , : . > 

NOTE : Portions of this field pilot were conducted by. 
John M. Sullivan of Sumneri Washington — '^ho 
was listed as second author to an earlier writing 
wrrting and documentation of this investigation. 



^Part III nisw prepares to address the issue of actually' 
operational izing the^claims, of the first and second parts. That 
is, can a^sti*uctured decisioning system be formi>lated to evaluate 
the specific criterion-referenced alternatives of various program 
units for fiscal roll-back in*a budgetary crisis; and can such a 
criterion-referenced,, multiple-alternatives model be uti^lized 
confidently in a funding deallocation situation? 

The author has had the distinct (thoug^unf ortunate) 
advantage of residing in a state which now fifw^ itself in the 
midst of a severe, financial emergency. In all^sctors of 
education, from the state policy level to the realm of the 
classroom teacher, alternatives are now being studied to brace 
for a cut to state-support for both K-12 and post-secondary 
education. To present the design and utility of the ROLBAK 
formalization, a single school district has been fleeted for the 
required piloting activities to demonstrate the ROLBAK 
f onfliul ization, 



Need for the Research 

In an age of expanding technology, the role of sophisticated 
appro^aehes to d^ecisionrniaking has beconle more accessible to the 
field administrator. Nothing supports^' this view more strongly 
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than the' recent advance of computer technology 1n particular.. * 
Yet, those Individuals who ^cou-ld. best afford the* adv.antages of 
•such sophistication remain the greatest obstacles to. the accep- 
tance of sophisticated tools as a beneficial, tooT for'"'d^a analy- 
sis and eva-luat-ibn. The situation surrounding, th& fuirding . 
deallocation of specific programs' is .a. clear example.. ^ 



' T. 



Scant, resources recfii.irel;reVisi*Dn of. expanding service. ' • 
.activities. Compounding the prpblem: of forced decline is the ; 
fact, -th'at many years of affluence in -the avai labinty of wide, 

\ diverse servtcedel^ery now c,louds the iss'ue , of .which- services' 
are assent i aV and- whi.ch. are . a' luxury that is,, the diff^rence' 

; between . entitlement On the' one hand,, and enrichment on the other. 
Therefore, the evaluation of current operating programs •for 
posslD^he eligiinatiorr (.or reduction) will not only require 
assessment of performance, but also a m^easure 6f the pribgram'^ 
demand and need. .As, the decision-maker adds the criteria of heed 
and demand to the ''f-lr^ady -generic criterion list of ' 
ef.fectiveness,- efficiency. Satisfaction and expenditure, the role 
of a multiple alternatives formulation to determine prograius for 
retention vs. reduction via an analysis .of multiple, competing 
criter.ia becdmas paramount. ' " ' 4 

.Finally, the, nefed for a demonstr.a^^ " 
referenced, muitiple-alternati yes decisioning model is'dictated 
by the par'al l.el .need of due-process . - Not only doel*the decisf;on 
maker need, to be convinced of the efficacy of a carefully for- 
mulated MAM framework, but the, program partTci pants themselves ' 
need- a firm und,erstanding of . the model ing perspective'. People 
affected by the itiodel-geherated solutions ( in this case, programs 
to be terminated) must accept that their personal, interests were 
part of the decision, and that the relevant criteria were taken 
.into account in the preparation of the final ^decisiort-. 
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Purpose of. the Research 

The mission of this undertaking is two-fold; first, to 
demonstrate the development and design of the multiple-alter- 
natives analysis framework foV the area of fiscal roll-backs; 
and second, to assess the relevant issues of decision validity 
and model reliability for the reader and potential user. We as 
scientists fully realize, that acceptance of our technique can 
reasonably come only after maximal critique and scrutiny. We 
have endeavored to step-by-step annotate the development of the 
ROLBAK mdoel for this particular study. And, we have employed 
the use of parametric statistical procedure^ in order to assess . 
the model's impact upon the task at hand, 
• ■ 

That task is this. Given an existing district program of 31 
individual and distinct units, and the costs involved — prepare, 
execute and evaluate the results of a mathematical modeling pro- 
cedure which utilizes a criterion-referenced base for determining 
which program units remain operational, and which program units 
must be discontinued. 

•The criteria involved represent the identified expenditure 
requirements of each program unit, -del ineated across the eight 
"object" categories of a program budget; and a single meausre of 
subjectiy,e'^ opinion on the part of central office administrators 
as to which units are more important than others. We limit the 
inclusion of criterion references to only nine indicators for 
convenience only . Many other measures must be included in the 
final determination of units to be deallocated. However, the 
demonstration of the model's utility will not require the loading 
of all relevant criteria into this piloting-formulated^ model . 
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Overview of the Research 

The outline f6r the contents of Part IlThave been 
constructed to accommodate a chronological discussion of the 
ROLBAK model's design, data construction, execution, and pos't-h*c 
evaluation. . - ^ 

The following section deals with the construction of the,, 
database for subsequent MAM-analysis. The next two sections will 
then present the rationale and methodology- for utilizing the T- 
normal transformation of the new-scaled measures (dollars of 
expenditure). In addition, a brief discussion of special con- 
siderations in dealing witti scant matrices will be presented. 

The fifth section deals entirely with the search for initial 
model feasibility — that is, the identification of the correct 
mix of constraint values (RHS) to permit an initial solution to- 
the' model. 

The next two sections present the results (solutions) deve- 
loped through the use of "restricted" and "relaxed" models, 
respectively. These two sections have been developed separately 
to highlight the differential impact of weighting. 

/■ 

The final two sections provide both a comparison of the 
restricted versus relaxed solutions, and a generalized discussion 
of the total ROLBAK performance under analysis. 



I 



299 

o302 



CONSTRUCTION OF THE DATABASE 

^ "y — s : 

■■ • ■ ■ p 

Altfiough an actual field application (i.e. "for re^l") of 
ROLBAK to a fiscal emergency would necessari ly include many. cri- 
terion references to effectiveness, efficiency, need, demand, 
satisfaction and expenditure; the authors have' limited the pilot 
of ROLBAK ^to a small aggregate of measures. Under the broad , 
title of 'database' will exist the jiumerical values required to 
^operationalize the functions of the constraint matrix, con- 
ditional vector (RHS), and the objective function. Finalljy, 
three distinctly different scales will he used "^o^demonstrate the 
versatility of the model's data-input r^,equirements. 

Source of Data ' ^ . ^ 

Data for the model's execution represents two generalized 
measures: (1) a, measure of expenditure requirement(s) , in 
thousands of dollars; and (2) a measure of subjective bias, 
ordinal ly-scaled in units of rank {i.e., 1,2,3,...). 

■ ^ The expenditure data is input to the model in two separate 
fashions. The first, segregated by object-category, provide 
eight (8) separate expenditure amounts for each of the program 
units under consideration. These object categories are defined 
as projected al locations for: 



1. 


CERT 


- certificated salaries 


2. 


CLAS 


- classified salaries 


3. 


BENE 


- employee benefits 


4. 


SUPL 


- supplies and materials 


5. 


INST 


- instructional supplies 


6. 


CONT 


- contractual services 
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7. \ TRAV - .travel expenditures 
. 8. CAPP- capital outlay 

\^ ; ' ^ ; 

These measures (originally in $1000' s) will be later transformed^ 
into T-normal scores.^ Secondly, a category of 'total 
expenditure s^ required will be input to the model. This parr 
ticular constraint will be utilized to efficiently control the 
'cutting bias' of the model execution.. 

The second, general input .mlnneT^TrTT'e^an^^ 
perceived expendability" attributed to e^ch of the individual 
program units. Central office administrators were directed to 
rank the programs ufider consideration as to their degree of rela- 
tive expendability, with 1 = most expendable. 

■■. ••,> 

For this particular ROLBAK pilcl^r'a total of sTprograms 
were evaluated to determine the membership"* of the target set for 
deallocation. The criterion indicators to perform the MAM analy- 
sis included eight measures of object expenditure and a measure 
of perce^'ved expendability , as well as a measure of composite 
expenditure. 



Method of Data Generation 

Total projected expenditures for each of the 31 'identified 
program units were delineated into 8 object categories, as 
available from district office budgeting records. ^lYie rank- 
measures of perceived expendability portray composites from the 
aggi^egated ranks of four staff members: superintendent, 
assistant superintendent, and two administrative assistants. 



In addition, the eigVt objects were summed to provide a 
measure of projected totan.expenditure by unit. The utility of 
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this composite measure will 'be discussed in a later section. 
Where no ^expenditures wereSloted Tor a frarticular program under a 
specific object, the v^lue of 0 (zero) was assumed. Such zero- 
cells form ^ a scant (or sparse)- matrix. ^ Necessary controls for 
'the analysis of. scant matrices are discussed in the succeeding 
two sections. - . _ * 

i 

Matrix Formating for MAM Utilization 

^Figure 8> displays the raw database to be trarisformed to 
T-normals (see next section) and subsequently evaluated by the 
MAM procedure. Note that the model will incorporate 10 criterion 
measures for analysis: expenditure by object (8), total expendi- 
ture by unit (1), and perceived expandability (1). As will be^- 
discussed in a later section, the total unit expenditure criteria 
will be utilized twice under actual model execution: once to 
establish a level of minimal cuts,>and the second to provide a 
upper bound on the modeTs ^cutting' (we did not want the proce- 
dure to go "wi Id") . .n 

Recall the reason for the database described in '<Figure 8>. 
These measures will guide the ROLBAK- analysis in determining 
which units wjll be allocated vs. deallcJcated funding — based 
not only upon their expected expenditure by object but also upon 
their de'^ree of* perceived expendability. In addition, the 
measure of total unit expenditure (across all 8 objects) will be 
utilized to control for determining when "enough cuts" have been 
made to balance the new budget limitations. 
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INITIAJ- t-NQRMAL TRANSFORMATIONS 



^ * One of the original :directions to this text was to" present 
the versaxirity of the MAM framework to accept a wide array of 
measurement scales as indicators for the values of the 'criteria. 
Although not discus.sed at length, all scales (i.e. nominal,' 
ordinal,' interval and ratio) can be accommodated by the MAM 
model. "'^ , ^ 

In addition,\/he MAM framework in general and ROLBAK in 
partitular, can bej^tructured to model one of two situations (or 
both) concerning measured impact: impact of t\ie system modeled 
specific to t|ie individual effect for each^ alternative's value; 
and.impact to [the system modeled generalized to the collective 
effect for all alternatives' values. Briefly, the need for 
control of specific, individual < effect (the former) addresses the 
need to measure the utility of each program alternative, and its 
absolute ability to coexist with other alternatives as part of 
the solution set. The use of a control for generalized, collec- 
tive effect (the latter) however, addresses a less rigorous need 
to measure the utility of a program., alternative, and its rel ati ve 
ability to become a-member of the solution set. 

^ Extensive research has been accomplished over the past five 
years by this author to understand the implications of a 
: general ized, col lective. measurement system, for crj-t^erion eva- • 
f luation and control . Specifically, this research has centg^ed 
about the usefulness of standardized l^normal ized) measures to 
accomplish this collective conty^ol need. Early work with z- 
' scores was satisfactory, but required vigilance for the arith- 
metic ifnpact of weights beneath the mean, that is, the negative 
values bf.z-score. Conversion to.T-normals precluded such 
concern, andforms the primary measurement scale for the object 
expenditures, in the ROLBAK model. 
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The use. of stanciardized meastjres allovys the decision-maker 
to measure the relative impact of .each criterion's weight < for 
each' unit alternative) without being concerned' -about the specific 
dollar amount. Reliaoce upon'relat'ive -impact via thej'object" 
criteria is valid within the ROLBAK^ system, since kn additiona.1 
^criterion of- total expenditure for each program unit is present.' 
' ■ ■ r ' . . 

Transformation Consider^ations for a S^ant Matrix 

Before proceexkuig witH specif ic i llustration of ' normalized 
transformation, we add a caiiilonary note Concerning data matrices 
with a high' number of empty ceaj5< Empty cells normally mean one 
of two things: either the meas-ure was 'zero*, and therefore a 
zero was entered; or the criterion was inappropriate to that par- 
'ticular alternative, and therefore no measure is/possible. As we 
said much earlier, the choice of relevant criteria which are 
applicable across all alternatives will preclude the modeT^ * 
builder from*the need to control for sophisticated confounded 
effects from irrelevant criterion variables. 



For ROLBAK, the amount of zero-cells d^onstr^ing zero-cost 
in particular objects for certain alternatives is vefy large — 
large enough 'to call the data matrix/a "scant" or "sparse" matrix 
(more zeros than not). To controX^or this situation, and to 
provide a better environment for the use of the RHS control 
values, we chose to exclude /he empty eel Is^'^'I'rom calculation of 
the normalized measures. 

» 

This is to say ( tyfease^ . read very carefully now), that: 



/ 

the normellized valuer associated with a particular 
cr1terl6n, demonstrate the relative weight of that cri- 
terion^for the individual unit, relative to the other 
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unit weights where such criterion expenditure actually 
exists. . 

u pin stop reading carefully now!)' 

\ i ' — ■' ' \ ' ' 

In effect, the zero weights denoting no expenditure are not 
part of the original distribution that will compute the standar- 
dtzed measures; and .thus the ^calculated weights will be more con- 
versant with the other>.unit values where cri,terion expenditure 
actually exists. \-- ' 



First ^Staqe Transformation to Z-Scores 

y * ■ ■ ■ 

The following subsections are presented in brief to he'lp 
those readers whtf have misplaced their statistics* knowledge (who 
has not?) . * ^ ' ' 

7 

f ^ ■ ' 

A z-score is a normalized measure, standardized to reflect 
the relative weights of each of the *raw data' values which f^m 
a specific distribution of ^scores (in our specific case, the 
distribution of expected expenditures, by object catagor^) • A z- 
score represents the mean of the raw distribution as ^ '0.00', 
and the standard deviation as a '+1.00'. That is, a .raw score 
which represents a single standard deviation above the mean of 
the distribution is computed as a +1.00. "If a score is one and 
one-half times the standard deviation bejow iihe mean, it Is 
repre'sented as -1.50; and so forth. 

For the zealots among you, the transformation formula for 
computing a z-score form a raw distribution is a follows:" 
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where, 



X (mean) 

■ " ■ ' r 

and, • ' ' ^ 



(standard deviation). 



subject to. 



1, = 1,2, . ..aN values, 



Second Stage Transform.ation to T-Normals 

The use of T-normals is simply suggested as- a useful tech- 
nique for circumventiT|[g the negative vlues of z-scores (or 
z-normals, if you wish). T-normals- are' standardized measures, 
with a mean of 50.00 and a standard deviation of 10.00. Thu4, 
negative T-normal would result only from ^a raw measure, whose 
value resides greater than 5 standard deviations below the mean 
of. the distribution (somewhat unlikely. In the usual case). 



T-normals are computed directly form z-scores, as follows: 
T. = lO.OCz.) + 50.0,' 

such tliat a z = -1.0 becomes a T = 40.0, a z = +2.5 becomes a T 
75.0, and so forth. ^ 
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FORMULATION OF THE ROLBAK MATHEMATICAL MQDEL 



The reader Is redirected to cFigure 7> to review the-format 
of the gener'allzed MAM framework. Recall that the model utilizes 
three distinct though obviqusly interrelated segments. The 
first, the constraint matrix,' contains the coefficients for the 
system of simuVtaneous linear inequalities (and equaliti-es), 
whose independent variables are the alterPfative progr^ 'units: . 
Therefore, these coeffic-^ent values are really the criterion 
measures associated with each independent variable's 
''performance" or "need". , , 

The second distinct segment, the<.conditional vector (or 
"right-hand-side"), contains the composite measures which 
restrict the summations. of the coefficients of the independent 
variables, as those independent variables are evaluated for 
inclusion v/ithin the final so"! uti on set. These RHS-values are 
the upper (or lower) bounds associated with the linear 
inequalities, and the ex^t standard associated with any linear 
equation. ' -V ; , 

The.Jast segment, the>p!bjecti ve function, provides the .'-^ 
guiding force behind the ieiection of alternatives for membership 
within the so-lution set . RemSfiber that objective function • 
(sometimes referred to as cost vectors, whether measuring cost or 
not) must be either maximized or minimized, .depending -upon the. 
objective of the problem modeled. Maximizing (or minimizing) the 
summation of the objective function, is refelrred to (in 
fashionable circles, of course) as optimization. 

Under optimal ity then, the goal of the model is as follows:. 

to formulate the "best" solution mix of alternatives . 
based upon the values of the objective function; given 

30C 



the constraints defined by the simultaneous system of 
^ inequalities (and equalitiesj as modeled by the 

c'Onstraint matrix, anb the limits provided by^the con- 
ditional vector, ; ^ ' ^ 

The Constraint Matrix . ' » ... 

~~~ ~ ^ 

To formulate the ROlTfeAK problem,A a total of 31 individually 
funded programs were defined for evaluation. Each program's ' 
budget was delineated into it's individual 'object expenditure' 
requirements (certificated salaries'^ instructional supplies, 
etc.). These initial eight expenditure breakdowns form the first 
8 constraints of the constraint matrix; and are entered as T- 
normal transformations. The next two constraints are identical 
veqtors containing the total, composite expenditure requirements 
for each program unit. Expressed in thousands of dollars, these 
two vectors will provide the basis for controlling the model's 
final, total amount of final deallocation. The last constrai,'nt 
vector, in the matrix, contains the ranked values for the 
"perceived expehdabi lity" of each unit; where 1 = most expendable 
and 31 = lea^t expendable. 

Thus the criterion coefficients of the constraint matrix 
represent three distinct measurement features: T-normals 
measured in standard units, total expenditure measured in 
thousands of ""dollars, and expendabi 1 ity measured in ordinak 
ranks. . 

Conditional Vector for a Scant Matrix 

The RHS-values of the conditional vector served to opera- 
tionalize the simultaneous system of the constraint matrix; that 
is, they establish the limits which the vector-coefficients sum- 
mations must comply with. 
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for^^e initial 8 object-expenditure constraints, the ^ 
RhTS-values are computed to effect a generalized impact upon the 
system as a whole trgatj ng^ a1 1 objectSseaually , .Thus, expen- ^ 
ditures projected in any one category m not place their assoc- 
iated programs^ in Weighted jeopardy*, .^Although In some modeling 
cases such weighting will be desirea]Dle, the current example 
.weights ^11 equally for demonstration purposes,' These specific 
values will be discussed at .length in the next section. 

The next two constraint vectors are^ictenti.cal in that their 
coefficients represent composite object expenditures for each 
program. The district's current operating budget comprised 893.5 
(thousands) dollars. The goal of the model was 'to develop a plan 
f orv^effecting a revised operating level of not less than 675iO 
dollars (lOOO's) nor greater than 700.0 dollars (lOOO's^). ^'To 
model this objective ^(constraint), the sum of the first vector 
was limited to 675.0 (greater than or equal, and ths sum of the 
^second vector to 700.0 (less than or equal). In effect, this 
"bracketing" allows a 25.0 dollars (lOOO's) flexibility factor 
for model evaluation. ' 



The final constr^aint, the measure of perceived expendabil- 
ity, was modeled to effect a smaller sum (minimized).. This was 
necessitated due to' the fact that a smaller rank represented 
greater expendabi 1 ity and thus, the sum of smaller^ values pro- 
duces a "preferred" smaller amount. ' • 

. The considerations . required for a scant matrix involve only 
the first 8 inequality vectors. However, the sum of zeros will . 
not deter from the utility of the conditional vector in success- 
fully controlling the sum of the remaining sums. Since the empty 
cells represent no expenditure for that particular object, the 
choice of the associated program unit willnot contribute to the 
RHS-requirement (limit), (Again, please refer to the next sec- 
tion for a more detailed discussion.) 
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. Cyclical Objective Function * . ' 

, Since the construction of the objective function', and^'the ' 
subsequent maxipiization or mi'nimizatiori of. its sum', defir>es.-what 
■we call 'optimality , the content to the- O.F; is a- biasing factor 
to the model's evaluation of alternative's. It is reasonable to 
expect a different mix of solution alternatives, if. the moqjel ' 
utilizes a different objective function changes- from maM\'\fa- 
tion' to minimization of the same objective vector. * 

' ■■■ V .■ 1, ■: 

- ' ' , , . . » ■ ; . .■ 

ROLBAK examines the effect ' such manipulation has upon solu- v 

tion results by cycling each individual constraint vector thrdugh* 
a separate execution as the objective function.' Moreover, the 
focus is alt^ered to investigate both optimality directions, maxi- 
mization and •■|i|f^^"z'at1on. ., 

■ The Problem - ■ ' ' ■ 

Given the structLl^'e of the MAM frcunework discussed above,, 
the resulting R0l||||^5ipdel will select:(X_^ = i) those program 
units to be reta0ec}^y|:Wa^ are to* receive funqling. 
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SEARCH FOR REGIONAL FEASIBLITY AS BENCHMARK 



' . . The tnitial attemfDts in "executing a M/^M-desigWd-^ 
r'eq*uires the; establishment of first , initial region feasibliry of 
the :decision space, and second ^ a benchmark from which th.e mani- 

'pulatvon of RHS-weighting and cyclical optimization cari both be 
measured. Decision sDace .(regional ) feasibi I ity simply means 
that at least one solution exists which satisfies the require^ 

;ments of the constraint matrix and conditional vector, If-no" 
solution exists, under any circumstance allowed by the linear ^' 
Inequalities and equal i ties, ' then the deci^ioning (constraint)- 
region is declared to be "infeasible"; and the model either 
•altered or abandoned. 

' ■ . ' , ..■ ■ • .' 

Once feasibi.lity is. determined, a benchmark is established 
■ ■ . ■ ' . ■ '-t' * 

to begin the, cyclical evaluation of the various agreed-upon cri- 

ter ion values:-. The benchmark may in fact be the initial point at 

which .feasibi 1-ity is .determined. However, serious practitioners 

of the^art . (obviously us!) will search f or. tv/o separate modeling 

conf igurations'from which to observe the;effect of the varying 

. opt irrraTity criteria. These separate configurations can best.be 

addressed '31$ states -of restriction ; and relaxation . • 

The restricted mod^l .contiin^^^^^ which force the 

execution to choose its soRiti on s^t most carefully; that is, the 
limits imposed are very restrictive as to what is allowable to 
constitute a solutionv. .'On .the;b^^^^^ the relaxed model uti- 

..Tizes such RHS-values as will .invite solution set membership pat- 
;t.ern5 'which '.yi^ely :diff/er. The..aut^ chosen both so as to 

, piease ' eveh " the" most .. skept leal .'.of our- readers . 

■,i > . ' ■ . ■ . 

■ r' ■ i'' 0 •■ ■■ ■■ » 
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*N' of the Scant Matrix 



The normal procedure in attempting to establish feasibility 
is to arbitrarily project the number (N) of solution which is 
likely to result from the successful implementation of the model. 
With the use of J-normals, and the given T(mean) = 50.0 and 
T(standard deviation) =10.0, the arbitrary N can, be used to 
establish a beginning RHS-value: 

N(50.0 + 10.0) = N(60.0) 

for a perceived upper bound; and: 

N(50.0 - 10.0) = N(40.0) 

for a perceived lower bound. (The rationale for such con- 
siderations has been discussed, at length in a previous chapter of 
this report.) 

The existence 'of a scant matrix however provides a rather 
unique:;situation concerning such 'N' formulation. That is, the 
N's concerning each criterion across ^11 alternatives will 
differ, based upon the number of empty (i.e. zero) "cells. And in 
fact for this particular ROLBAK formulation, this is exactly the 
case. Referriqs to Figure 8 (on page ), 'you will see that- the 
number of non-zero cells are as follows: 
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Under these circumstances, the useful relationship of 



N(T(Mn) + T(S.DJ) 



must be changed to 



Nk(T(Mn) + T.{S.D.)) 



for each separate k = 1,2 



8 of the object expenditure 



categories . 

Expected Solution Index 

The expected solution index (ESI)^ controls for both the 
existence of a scant matrix of zero-cells, -and the necessity to ' 
investigate varying levels of solution N*s — that is, the number 
of units which may be members of the final solution set. 
Although inextricably related, we will develop each separately 
for the sake of understanding their unique contribution. 

The existence of a scant matrix will provide a varying 
number of non-zero cells. Tooperationalize the utility of the 
N(T(Mn) + T(S.D.)) idea, we must vary the N for each computation 
of the particular RHS-value. Furthermore, the region of feasible 
solution(s) will likewise require the search for a suitable 
(expected) solution set size; that ^s, to allocate (for example) 
funds to 10 programs; or .12; or 14; etc. 

The EIS is calculated to take into account both scant matri- 
ces and varying solution set membership by uti 1 izing. the postulate 



(N > 0)(E) 
N(total) 
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where. 



(N > 0) = number of non-zero cells for the given 
criterion'- constraint; 

(E) = number of expected solution set 

alternatives; and 



N(total) = number of total possible alternatives. 
In our ROLBAK example, this expression can be reduced to 

- ilj . 0) (E) . 
31 

For. example, if we were to examine the ESI for the "certificated 
salaries" constraint (12 non-zero cells) and an expected solution 
membership of 10, the index based upon the (M > 0) would be calcu- 
lated as: . 



N>0 12 



and with the expected membership (E) of 10, 
N>0 

31 (E) = .387(E) = .387(10) = 3.87 

This index , and its relationship to the T-normal values will -be 
described in the next subsection. 
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RHS-- Values by Index 

<Figure 9> suirthiarizes the calculation of RHS-values utilizing 
the idea of an expected solution index (ESI) for a scant matrix, 
for example, given the constraint of certificated salaries 
(CERT), with 12 non-zero entries and- an expected solution mem- 
bership of .10 units, the RHS-value would be computed as: 

12001 ^gQ^Q -^10,0) = 3.87 (40.0) = 155.0, 
31 

assuming that a "linear bound" is the desired RHS intention. 

The RHS-values for PERC and COMP are arrived at arbitrarily 
as well, but without resorting to the above scheme for T-normals, 

Search for Feasibility ' : 

Use of the ESI system discussed in the preceding subsection 
established immediate- feasibi 1 ity, with concurrent values for 
PERC and COMP as shown. Any value for PERC less than 500, 
however, lost the feasible region. 

Search .for Benchmark 

The range of expected membership values was varied from E = 
10 to E = 15, and the def inition(s) of relaxed benchmark attached 
to: ' . i 

■•; E = 16; PERC; = 500, \^ 

and restricted benchmark attached to: 
' ^ . E = 10; PERC = 600. 
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Figure 9. Ccwnputation of Conditional , Vector (RHS) Values for Constraint Matrix 
with Zero Sub-Matrices and'Cell-Entries Based Upon T-Hormal Scores 

RHS-Values by Expected Inrlnr** 



Criterion Constraint 


Code 


N>0 


INDEX* 


E=10 


E=12 


E=14 


L;J6 


Certificated Salaries 


CERT 


12 


.309 


155 


186 


217 


248 


Classified Salaries 


CLAS 


10 


.323 


129 


155 


181 


206 


Employee Benefits 


BENE 


08 


. .258 


103 


124 


145 


165 


Supplies & Materials 


SUPL 


14 


.452 


181 


217 


253 




Instructional Supplies 


Tnst 


10 


.323 


129 


155 ^ . 


181 


206 


Contractural Services 


CONT 


14 


,452 


181 


217 


253 




Travel Expenditures 


TRAV 


02 


' ,065 


026 


031 


036 


041 


Capital Outlay 


CAPI 


18 


.581 


232 


279 


325 


372 










Administrative Perception 


PERC 


31 


(Restricted = 


500 / Relaxed 


= 600) 




Composite Budget 


COMP 


31 


(Lower Limit 


« 675.0 / Upper Limit = 


700. Q) 



N>0 

* Index » "TT" 

** RHS (EXP) =» tN>0] P(Mn)+ T(SD)1 / 31 - (Index) (TmHO). where ; 1. -Index (40) for Lower Bound ()|) 

^ _ . 2. Index (60) for Upper Bound (|) 
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Since. the object categories were constrained to force sum- . 
mations greater than or equal to the RHS-values established by 
the ESI, the larger. the RHS-value, the more difficult. to find an 
acceptable solution ~ therefore, the more restricted. Likewise, 
for the relaxed system and smaller values of the RHS. 

For the purposes of the remainder of this chapter, the 
restricte'd and relaxed benchmarks will be utilized to observe the 
effect?;^ of cyclical optimization upon solution set membership. 
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CYCLIC OPTIMIZATION OF THE RESTRICTED MODEL 



The first of two major quantitative assessments, the cyclic 
optimization of e^ch of ten (10) criterion linear (convex) com- 
binations as the objective function, produced analyzable results 
under both maximization and minimization. This section will 
study these results, and address their relationship to both the 
model's execution and the criteria utilized, for the restricted 
model. 



Maximized/Restricted Solutions 

<Figure I? displays the results of ^the various Cyclical 
maximizations within the restricted setting. Of the possible 
combinations of the available 31 units for solution membershh'p, 
only two distinct solution sets were^formed. The mix set of 10 
entries 

. [01,02,03,04,05^07,09,11,15,17 3 

produced a new budget of 680.0 dollars (lOOO's) for a savings of 
213.5 dollars XlOOO's), in five cases. Similarly, anothe five 
instances formed''the mix set of 10 entries 

[ 01,02,04,05,07,11,15,16,17,23 3'^- 

producing a new budget of 680.5 dollars for a savings of 213.0 
dollars. . 

Additional technical data has been included within the 
figure for the more technically knowledgeable. - 
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Figure 10. effect Uoon Budaet Oedllocation Oecisions Based Uoon the Variable Flows of a Cyclic Objective 
-Function, jno tne Interaction of a "Maximized, ^^estricted" Constraint Iterative Projlfein. 

Objective • ya.timig^tion Constraints: ^estrtctea • 

(tXP«l6; riAC ' 500) 
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217.9 


433,9 


330.0 


362, i 


50.0 • 
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IIS 


228 


27 


114 


51 


5000* 
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530.0 


680.5 


680, S 


530,0 


630,0 


630.5 


6SO,5 






(- :ut> 




-213.5) 




(-213.5) 


(-213.0)(-213.01 
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note: Total Initial Suooet - 393 . 5 (SlOOO's) 
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Minimized/Restricted Solutions 



Minimizing the various objective functions within the 
restrictive setting produced similar results as displayed in 
<Figure 11>. Four occurrences of the solution 'vector 

C 01,02,03,0.4,05,07,09,11,15,17 3 - 
and three occurrences of the solution vector 

- C 01,02,04,05,07,11,15,16,17,23 3 • — 

resulted in the minimized, restricted setting. Unlike maximized 
optimality however, the use of minimized objective functions 
failed to produce a solution in threes separate instances. 

Validity Evaluation of the Restricted Model 

Analysis of variance procedures were utilize^d to detect the 
extent of criterion difference between membership in the solution 
vs. non-solution sets. Since ^optimal ity within the restricted - 
setting produced only two different combinations of solutions, 
these post hoc assessments, were easy to execute. Results are o 
presented in xpigure 12>. 

A review of the ANOVA results show that in all cages except 
one, the mean values of the "included" criterion indicators were 
greater than the non-solutional weights; arid were therefore con- 
sistent with model expectations and formulated constraints. The 
one exception occurs in both optimality settings when the percep- 
tion of expendability was used as the O.F. . ' 

It is also interesti^ng to note, that the 
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Figure 11. Effect Uoon fludqet Uedllocatlon Decisions Based Uoon the Varieble Forms of a Cyclic Odjectv/e 
Funccion, cn* Interaction of 3 "MinimijKO, ricstricceJ" Conscraint Inc^rative Protjlem. , 
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rigure 12, Summary of Cyclic Optimal ir.y Oirecrions (of Criterion Objective 
Functions) Utilized in Guiaing. the "Ful ly Restricted' (E;<P=16; ' 
PE?JC=500). Problem to Two Distinct Solutions; and Res'ultino Object 
Expenditure Impact. ■ ' 
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[ 01, 02, 03, 04, 05, 07,09, 11,15, 17'3., 

pattern resulted |n six (6) statisticaTly significant differences." 
while the other .pattern resulted in only five (5). Though, barely 
different in number, the criterion producing such differences (as 
'O.F.) varied in bot+^ases. The reader should recall that signi- 
ficant differences in criterion mean weights portray the ability 
of the model, to utilize such criterion constraints within the 
decisioning and. solution set building process. 



Reliability Evaluation .of the Restricted" Mbcjel^ 

J-.Disc.rimt|Tab^^^ analysis was ernpioyfedYjO: study the 

cofTsistency and predictability of the model's Vfunc.tii on in pro- 
ducing reliable solution- sets . > ''^ 



1 



set 



<Figure 13> displays the discriminant results for' solution 

' C 0U02,03,04,05,07,0|,'11,15,17 3. 

The major criterion values predictive of the estabTfshed solution ' 
is shown in the order of their importance. Re-predict1on wcis 
established with 96.77 percent aG£Hjrady. 

The results for tllQi solution set - 



A... [ 01,02,04,05,07,11,15,16,17,23 3 

are found in iFigure 14>. .Irv this case, orily three critarion ^^<. 
distrlDutions were required''^.b re-predict membership at an f^jj^*' 
equivalent 96*77, percent ac'curacy. . 

.... . ^ %;0'-^. 
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% rigure 13. Use of Dlscrl.-nlnant Analysis in Predlcti ng" Program rnclusion for 
; ;• Sucgstary Revision, Solution =1, 3ased Upon the Cycl 1c Ootlmizatlon 

;•■ -i"; ». ., OT ^ne ^.2Strictsd''Rroolem. • 
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Figure 14, 'Jse of Discriminant. Analysfs in Predicting Prcqrim Inclusion 
• 'ipr 2Ldaetary Revision, Solution -2, 3asea 'Jpan cne Cyclic 
: - Opcimi zacion of ?:he .'Restricted' Preble.^. 
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■'7 Summary of Criterion Value., i n ' Qi scrlrninad no Incius'on Oecisicns 
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' '-^^ f ; ■■H:-& ^^klC OPTIMIZATION OF .THE'' RELAX ED MODEL 

The second of the two major quantitative assessments, the 
relaxed setti ng'.pr.oduced a wide diversity^of solution sets. In 
fact, out of twenty executions and seventeen successful 
feasibilities, a!ll seventeen solution sets were' uUtque. 

Maximized/Relaxed Solii;liviib]f:is, ^ 

<Figure 15> dispj;^ys ten unique solutions, one for each of 
the cyclic opt imizati'pns under ^maximization. All solutions were 
successful in rebudgeting between the 675.0 and 700.0 limits. It 
iX^perhaps more interesting to study the column of number^;- 
labelled 'seleqtj'on tally', oh the right side of the figure. The 
repetition vijijiyj^hich particular units were chosen for continued 
funding resemfe'le's closely the two. solution sdt§^ constructed with 
the restricted model formulatiQh;^ ' ' 

Minimized/Relaxed Solutions . ' 



V-.' Minimizing in the relaxecj.^ failures at 

set building. Of the seven solution sets/ constructed, all are 
distinct; .and different from the friaximization sequence. 
<Figure 16>' presents these data results. '>^. 

The alert reader willvalso note that the relaxecf^^setting 
produces varying nu{T!b^rs of units within the solution set (low of 
10 to a high of^^jPS^Mits selected). ^ 
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I ijii. Iii. iM.xt Upon butJjcl O^anocaliou iK^liiuMv UjvcJ Upch lliy Vjrjjlile FIo»<i uf 4 Cyclic Objective. 
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'Validity Evaluation of the Relaxed Model 



<Figure 17> and <Figure 18> contain the analysis of variance 
results concerning the mean values for criterion weight 
membership. As might be expected due to the diverse membership 
of the many solution sets formed under relaxation, statistical 
significance is not as controlled and patterned as the restricted 
model ing outcomes. 



] 

Reliability Evaluation of the Relaxed -Model • . 

Because of theTeventeen different -solution sets formed by 
opt1miza1;ion within the relaxed setting/ post hoc assessments of 
consistency were undertaken in a different fashion^ th^n those 
under restricted optimal ity. As <Figure 1'9> and <Figure 20> 
demonstrate for maximization and minimization respectively, the 
frequency of a unit's selection as a solution was utilized for 
discriminant analysis. Such a chojce to uti I ize frequency; 
obviously increased the interval variance of the dependeRt-v.; 
variable; and it is thus expected to diminish the extent^of -j^^^^^^ 
predictive accuracy. - •^;:;vh 

Maximization discriminants required five of the available 
ten criteria to predict membership at* 70.97 percent accuracy. 
Correspondingly, the minimization discriminants required six cri- 
terion indicators to re-predict at 33.87 accuracy: - V. 
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Figure 17. Tests for Level of Satisfactory Significant Olff-erences 82tv,een.0b jects of Budget 
Revision ( Included ) and Oel located SuUqets (Excluded); Relaxed MaxliT^ization. 
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■^») « CtJtisticjl significance of inean diff ertjnces. incluo'e vs. exclude. 

' "e-n buaaet- iiount for Includoa object or budget revision. 
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'1 :r 3 < .10 occurrences, iner;; criterion constraint values reflect cJesireable mean-value weigftts , 
jcros: Lne cyclic objective functions. * i --V 
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iir 0 < .\0 occurr'joces, wnere each defined objective function produceo desircable (nedn-vdlue;>«H^ghWT 
lin'li inn criterion constraints. . 




Fiyure 19. Use of Discriminant Analysis for Predicting the Fruquuncy of Oudyut Selection 
Rcsultiny from a Cyclic MAximizjUon of the Rulaxud Problem, ' 
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Figure 20. Use of Discriminant Analysis for Predicting the Frequency of Budget 

Selection Resulting from a Cyclic Minimization of the Relaxed Problem. 
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Summary of Criterion Value in Qiscn'minatinq Selection Freauenc i ? 
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COMPARISOr) OF THE RESTRICTED VS. RELAXED . 
DECISIONING framework " • 

V 

Modeling within the restricted setting produced the most 
're-predictab^^e' and ciriterion-signif leant results. Less cri- 
terion measures were required to explain the solution set " ; 
membership. And, restricted optimizations tended to require no * 
greater modeling effort l:han the relaxed setting (measured via. 
iterations and computation seconds). 

Results of the relaxed setting, however, provide a strong 
preview of the flexibility of the model for determining a wide 
array of solution memberships based upon varying standards, 
(objective function values), in addition, the relaxed setting 
also presents a hint of the diversity in model building based ' 
upon the weighting of particular criterion indicators by relaxing 
certain RHS-values whi le retaining others' in a restrictive 
fashion. 

Finally, both optimal ity sequences demonstrate the utility 
of the MAM system in general (and the ROLBAK ^system in ' 
particular);/ or evaluating multiple criteria, and selecting a * 
distinct solution set from among multiple competing alternatives. 

Effect of the Restricted Environment Upon Optimal ity 

The restricted environment which constrained the ROLBAK ' 
decision-making was constructed using an expected solution index 
(ESI) of value 16; and a perceived expendabj lity value of 500. . 
That is,/the design of the solution set (program ^jnits to be- / 
funded, for a total new budget between 675.0 and 700.0 (1000' s 
dollars); and reflecting an administrative perception of priority 
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for expendabi Mty— was required to exhibit the qualities of 
potential; solution with ^ pes;;S|;ft]ije. " units for 

each of the 8 object expend^.^ur^f€^^ modeling the indivi- 

dual programmatic budgetsV'VSe^^ each constraint was v;;::. 

cycled through t^ie/irtodel as the objective function (first for'^H'r 
maximization, then for minimization) in order to different ialTy^^^^^^^^^ 
direct the construction of the solut-j^on sets under optimal 1ty|f^^ 
that is, those solution sets which best, represented the 
.constrained environment design by the restricted, linear ine- 
quality constraints, and furthermore provided the most maximal 
(or minimal) summation of the objective function vector. 

* Optimal ity under maximization . -'Utilizing restricted 
RHS-value(s) vectors .to construct a,feasiblity region for ROLBAK 
decision-making, 2 distinct solution sets were formulated by ' 
separate sets of 5 of the available 10 cyclic objective 
functions. Solution #1: ' ^' ' i 

[ 1 2 3 4 5 7 9 11 15 17 3 

presented _10 program unit budgets for funding under the reduced 
budgetary levels, out of the existing^ potential multijDle 
alternatives. The 5^ object, expenditure (budgeting) vectors wh*ich 
produced these solutions under maximization were: 
■ • / . 
K CERT (certificated salaries);. 

2. CLAS (.class-Jfied salaries); 

3. SUPL (supplies and materials) ; 

4. TRAV (travel expnditures) ; and 

5. ;.. CAPI (capital outlay). 

A totaTo^ 213.5 (lOOO's dollars) was cl^^from the original 
budget of 893.5 (lOOO's dollars), deallocating 21; program units, 
resulting in a new, system operating level of 680.d (lOOO's 
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dollar^)'.. The other disti net solution set constructed under -/ /V 
maxiiTiUatlon, solution #2: ^v... ^ J • --^^ 

. " C 1 2 4 5 7 11 15 ; 16 17 23'J -' 

.also presented 10 ^program unit budgets for continued fundhig.rout 
■;,pf - the; potenti ai:,|^^-a1 ternati ves available. The remaining |'' 

object expendltcire vectors which pro.dUce'ci: these; solutions under 

maximization, were: 'V.:^.' 

^ ■ . ■ ■ i " ■ . . ■ 

1. BEN£ (employee benefits); ' . ' . ; ; 

2. rN;St\(instructional materials),| r//'^ ' 

3. CON?A(contractual services); ' 'L. 

4. . PERC '(administrati ve *perc^ v - • ' 

5. COMP (budgetary composites). - Vv'^- 

A total of 213.0 (lOOO's dollars )^'was cut from the onjginal 
budget of 893.5 (lOOO's dollars), deal locating 21 program units, 
resulting in a new, system operating level of 680.5 (lOOO's 
dollars). Thus the difference between the two solution sets, was 
approximately (lOOO's dollars ),3d 4 varying uni^tr^member 
slips.- ^^'"^'ir . ■ ' 

■ ' ' \ 

Optimal ity under minimization . ,.Ut,ilizing re/tructed 
RHS-value(s) vectors to construct the feasibility region for 
ROLBAK decision-making, the same Z.'clrstinct solution set were 
found under minimization, as^ were developed under maximization. 
Differences were observed however, both in the number of 
occurrences of the solution set, and in.,the;.objective function(s) 
which guided the solutipnal design. Sol-a^Qn #U 

. - I 1 2 .3 4 5 7 9 11 15 17 i 
resulted from the following 4 objective function vectors: 



1 . ' BENE V ( enipl oy^e'^ -benef i ts ) ; 
'--j'f^v^'. '••"■^^^ materials); 
• ;^ services|vvand ^ \] 

}\::-^-}::'''-A^.'r' ^ PERC ( ad(min;i5tr at1 ve. pergerfti on) . • ' 

The^rea'der: )vi [T no'tey:^^^ maximization, these same four 

vectors . collaborated on a different solution set. The resulting 
expenditure reduction of 213.5 (lOOO^s dollars) remains the same 
of course. ^.'Solution #2^ under minimizatiyn: 

V V- ^ V-.''-' ■ : . , V . 'V 

2;. . 4 5^ 7 Vn 15 16 17 23 :i ■ 
'". • > ■ . ■ . . , . 

■qccurred in ^ instances; ;ujnder the use of the •cyclic objective ■■ 
;functions: Z^:- • 

.4/ CLAS ( class if ied^'.s^llaries^ - "^z 

' "\ v \. \. SUPL (suppl ies ^ind;;mat.en:ali)^^^ and'^ ^ v .■ . . ■ 

The reader will ■•also note, that previouilj>'^unider maximizat^^^^ 
these same three vectors collaborated on a .dtff erent .'s 
set. As before, the resulting vexpenditure reduction -of .21:3.0 
.(1000' s doTlarsf(-r^ ' ^ :/ ■ 

Validity" ariaJiy's is of restricted results . For the purposes 
of this study, validity tests represented the administration of 
post hoc analysis 'tv determine if the resulting solutions 
reflected the original objectives of the ROLBAK model. The ori- 
ginal ROLBAK objectives v;ere forrtula'ted via the- construction of 
the linear 'object category' vectors. . Validation under these 
circumstances proceeds in two stages. Stage 1 validation is 
moot, since the' exeotfted ROLBAK model produced at :Teast one solu 
tion vector (in bi/rvcase, tv/o dis.tinct alternative solution 
sets), in conformkace with-p^e-defined RHS-- vector values. Stage 



■ 2 yalidati on proceeds to 'analyze the values -bf the .varioa?>< — 
constraint vectors; •and to test their "mean-difference's dfeermine^ 
by their solution versus the nbh-solutton membership. A ' 
Parametric,^ .oneway ^analysis of variance procedures were uti lized 
to test these, criteribn/'m.ean-value differences. .Solution #1: 

. . . • . o .;■ . ■ ■— ■ 

r 1 . 2 3 -4 5 7 ■ .9 - 11 15- 17 i . 

^demonstrated 6 .of the 10 criterion vectors -to pbduce statisti- 
cally ■significant fp. < .10) greater criterion, mean-weighis for 
the solution sets; then existing within: the non-solution set. 
The. 'six criteridn vectors werei . 

. I.-,; CERT- ^ 

° ■"2:;; (classified ■■ - > ..' s 

' ;;3r/gSU : -W ■(' 

4. - I;NST (nristructional' materials); ^ ^ z^' 

5; CAPI (capital , outlay); and ■ " ' ■ " , . 

6 . . ^^^^^^ / M^^^ • * 

Of. the remal n 1 ng . 4 vectors , employee benefits BOE ),' con.tr actual . 
services (CONT) , and' travel .expendltur^^^^^^ (TRAV); 'tht lack' ot p- <_ ; 
. 10 signlf 1 cance is not' yT.ewed as" : an Indicatii on pfpotentl al ° 
Invalidity^ due to .thk meVn-trends; observed:' - The relatively con- 
... f ounded p- 1 ev!£l, f or ■ ,adm1 n is.trati ve' per ceptpn ( PERD) -of- p ' =; .67 , ' 
... is ■un'derstanjable based -upon the ordinal scaling for PERC, in " 
which/each o)^d1nargraduat1on 2, ,3, 31) Is r^'pVesented. 

Sim)/iarly, solution #2_^'s'r ; - '- ' . 

C. l- ' 2 4 J ..7 . ,11' M5 15/17 23 J . 

. ■ . ■ ■ ■ ■ .-. ^ ♦ • ■ . ■ • , ; 

demonstrated ;5 of the\10 trtter-lon vectors ^'^^ statrstr- 
callj-S'ignif leant (p\c -,i^^ greater criterion; mean-^velghts for 
- the splutlon-set. ^ Thes^'fiWe criterion vectors-^we^: \ - .'■ • 
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• 1.- SUPL (supplies and materials)-; ^ ' ' . 

2. -rNST (instructional materials); 

:■ '3r.. CONT (cont;r>actual services); 

V 4./.V CAPI (capit3i,s?^utlay); anCl ■ 

5. • COMP (budgetary composi^tes) . 

The., remaining five .criterion.ifnean weights are acceptable, though 
not at the desired p< .lO'levisr: Much of Jthe iriabil'ity to .g^iln . 
the desirable.'p £ .10 level can be attributed to the large' pro- 
^pprtion of zero-cells (-scant' index) within the constraint matrix. 

( Reliability analysi^>:.Qf restricted resunts ; Foe. the pur- 
poses of this study/rel iabi lity-,tests represented -the admin- 

. istratloh^ of post hoc analyses to "determine if the Y,esul ting 
solutions' were 'predictable' based upon the"'multipTe data distri- 
bution configurations of the criterion vectors^ that is;, whether 
a particular program unit's inclusion (versus exclusion) within " 
the solution set was predictable. • . Parametric discriminant func- 
tion analysis procedure- were uti lized to evaluate the 'extent of 
such predictabil ity.*: -in order to predict the original sblution} 

,l|t -#l: "^'.L '■■ ■ ' ■ 

i'l 2 3 4 5 7 9 . 11 15 

a total of 5_ criterion distributions were required. Listed in 
the order of t^^eir importance (i.e., amount of variance explained 
and order of entry into discriminant construction), these cri- 
teria are: 



1. 


COMP 


(budgetary composites ) ;•■ 


2. 


SUPL 


(supplies and materials); 


3. 

'A 


• CAPI 


(capital outlay) ; 


■4. 


CERT 


(certificated salaries); and 


5. ■ 


CLAS 


(,classif-ied salaries). 
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The- di-scriminant re-predrction (recl'dss'if icatfon of solution set 
. membership) reSu.tted in 1_ mis-incTustofi /or a Tinjil ' 96 '.77 percent 
accuracy (repredic'abi ] ity) -'factor . In turn, sofution iZ: 



0 



^ . [1 ''W: 4^ 5.. 7 i5 ^16 I?'" '^23 J ' 

required the fallowing 3 ccit^rion di^ri buttons' tn^ orJer to pre- 
dict membership within t^e solution ^e1i' (In-order of 
^ importance^entry): ^ ^ ^, , ^* 

. ■ • • I " .'^ , • • "i* , ■ »• 

. 1- , '.COMP .(iDudfietary composites) ;^ ^" 
' ■ ^ 2. ' CONf\^( contractual services) ; and ^ 

3. * INST (instructional materials).. ' 

. The discriminant re-prediction for the second solution formed 
..upon restricted optimality resulted in 1_ m1s-e:»:clusion for ' a 
*final 96,77 percent accurSicy factor. The >eader wi 1 1 note, that 

the criterioii distribution COMP was the only vector utilized in 

both discriminant formulizations. 

Non-solution results . While maximization (optimafity) pro- 
duced a solution set for ^ach cyclic iteration of the various 
cr.nterion vectors, minimization was unable to* produce a solution 
vector, when the criterion vectors being ■^'minimizfidV were the 3^ 
vectors :^ ' ' 

1, CERT (certificated salaries); -^f 
. 2, TRAV (travej' expenditures); and ^ 
'3,* COMP (budgetary composites) , » ' * 

. Non-sotutions based.;upon TRAV can l^e'^discounted based upon tibe 
high proportion of zero-cei^l entries (29 of 21 Possible, cells . 
equal to 0);'in which case, the. model co\jTd not' 'fT>ake up its 
mind' Non-optimalit^ under the guidance of< CERT and/or COMP 

f <. 

„- . . ♦ . < 



however, is an interesting result. Precisely stated (and hope- 
fully in English), neither the CERT nor the COMP vector(s) could 
swmmate to a small enough final value (minimum); such that thie 
optimal objective function vector could physical.ly. pass.-through 
the feasibility region geometrically constructed via the V 
constraint matrix inequalities. (The authors apologiz^fpr. the 
last statement! ) . . 



Effect of the Reiaxed Environment Upon Optimality , 

The. relaxed environment which constrained the ROLBAK. ' 
.decision-making was constructed using an expected solution , index 
(ESI) of value 10; and 'a perceived expendabi I i'ty value of 600. ' 
That is, the design of the solution set was required to exhibit* 
the qualities of potential solution with 10 possible "average" 
member units for each qf the 8 object categories used in 
constraints. The reader will note, that since the RHS-value for 
administrative perception (PERC) was increased to value .600 , 
program .units wi'th greater 'perceived e)(pendabil ity * levels could 
still become members of the solution set -- that is, refunded for 
continuation. As with" the restricted environment discussed^in 
the preceding section; each constraint was cycled,;.through the 
relaxed model (sequentially) as the objective function. 

Optimality under maximization . ROLBAK .produced a distinct 
.solution set for each of /the 10 cycl ical objective functions uti- 
lized during optimal* maximization of the relaxed model. In fact, 
only the, program units: / * . , 

C 8. * IZ 20 -24 25 31 3 . " 

were never included in at least one of the 10 individual solu- ^ 
tion vectors. Budg^ary savings ^ranged from a Id^w of 193.5^ 
(lOOO's dollars) for the solution Vector: 
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CI 4 5 7 9 11 16 17 23 26 29 3, 



to a high of 218.0 (lOOO's dollars) for each of the solutions: ' 

CI 2 3 4 6 7 9 11 15 16 17 23 3 

based upon maximal CAPI (capital outlay), and: 

CI 4 5 6 7 9 14 15 16 17 18 '23 283 

based upon maximal PERC (administrative perception). The size of 
the solution vectors ranged from a low of II units to a high of 
.13 un.its .(as compared to the stable 10 units under optimal ity in 
the jfestritted system). ' \ 

Optimal ity under minimization , A fotal of 7^ optimal solu- 
tion vectors, each distinct, resulted from rrii-nimization within 

' 'Pi' 

the constrained, relaxed sp-ace. Th,e' criterion .vectors producing 

optimal results were: ' 

* . . ■> 

/ 

1. CERT (certificated salaries); ' 

2. SUPL (supplies and materials); 

3. INST (instructional materials}; - * 

4. CONT (contractual services); 

5. TRAV (travel expenditures); \' ■ 

6. PERC (administrative perception); and 

7. BENE (employee benefits). ' ' 

Programmatic savings ranged from a low of 202.0 (lOOO's dollars) 
based upon TRAV: 

[ 1 4 5 6 7 11 15 16 17 21 .26 3 

to' al high of 218.5 (lOOO's dollars) based uponyCONT: 

'■ ■ 343 ^ 
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C 1 Z 3 4 6 7 9 15 17 18' 23 26 3 . 



Unit membership ranged from a high of 13 (INST) to a low of 1£ 
(PERC). Not one of 7^ solutions under minimization was identical 
to the 10 solutions under maximization. " 

Validity analysis of relaxed results . The approach to vali- 
dating the resiJlts of th ROLBAK execution under:, relaxed condi- 
tions differed from that previously discussed wi%hin the'^ 
restricted state. Since 17 distinct solution^ sets were formed 
based upon both maximization and minimization under relaxed 
conditions, validation of the effect of solution set construction 
upon individual criterion mean-weight differences was- effected in 
two related ways. First s the frequency of p <^ .10 occurrences, 
where each defined objective functton (CERT, XLAS, COMP) 
produced desireable mean-value weights across the criterion 
constraints was explored, utilizing (as before) oneway analysis 
of variance procedures. These results are indicated as: 

[• \ (p < .10) 3. 

Secondly, the frequence of p .10 occurrences, where criterion 
constraint values (CERT, CLAS, COMP.) reflect desireable 

mean- value weights across the cyclic objective functions, were 
studied; and indicated as: .' 

C \ Cp < .10)- 3. 

<Figure 21> sumnariz^es these and Nq summations for optimality 
results under both maximization and minimization. The - 
Np-^frequencies . are analogous to those previously defined for the 
restricted environment. Based upon the computed percents for the 
tci^l frequencies possible, 10 and 7^ (for maximization and ^ 
mi|iimization, respectively),, the Nj^ values appear relatively 

?, ' '347 '. 
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across :ne cyclic objective runcttons. 
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identical; likewise for the N^. frequencies. Summing both the 

\ and Hq values, and ranking those sums (where 1 = high and 10 = 
low), an ordinal measure of relative weight can be developed. ^ 
Finding the absolute value of the difference between these sum ^ 
(N(^) and (Nq) ranks (i.e. j DIFF-RANKS | ), presents a measure of' 
relative consistency between and Nq values. The authors have 
previously thought that the greater the consistency, the ^greater 
the resulting value of the particular criterion vector. Thus, 
the smaller the rank-difference, the more valuable the criterion 
involved. However, careful examination of the ranks of Nj^ and 
demonstrate^ that the correlation between the two vectors of 
rank to be non-parallel (correlation (Nj^, m^.) = -0.666). And 
furthermore, that the correlation between the and Nq values, 
.and their difference (DIFF-RANKS) to be nearly non-existent' 
(+0.129 and -0.048, respectively). Further study is required in,;; 
this -area to study these issues of consistency and utility: 

Reliability analysts ^ofr^^l^ results . As. ;wi tii [ res^tr i cted 
results, discriminant ftincti:6fe ihe^ , 

predictablity of the'^obtaihed^olution vectors: For the*" relaxed 
environment however, membership in any particular solution • set 
was not the dependent variable; rather, the frequency^ of ''each ' 
individual program un>t being chosen 'for refunding across all ' 
criterion objective function^ (i.e. the sel'ection; tal ly f or the 
tracking matrix) was used as/ the dependent (.tq b6''*predicted)v 
.variable. Under maximization, ishe re-prediptio'h .of .the select i-^^^^ 
■frequency (i.e. the N of inclusion across 10 ^sucqessf ul '-^.---i'/^::: 
executions) required _5 criterion distributions -to formul/ate; the, :' 
discriminant functions. In order of entry^alfd importance, vthe^^^^^^^^^^^ 
were: * . . a ' 

1. COMP. (budgetary composites);/ \ 
^. INST (instructional materials);. 
3, CAPI (capital outlay); , — 



4. 
5, 



PERC (administration perception); and 
CLAS (classified salaries). 



The discriminant re-prediction (reclassification of total inclu- 
sion frequency) resulted in 4 over-estimates and 5 under- 
estimates for a final 70,97 percent accuracy (repredictabi lity) 
factor. Minimization results on the other hand required .6 cri- 
terion distributions to formulate the discriminant functions: 



1. COMP (budgetary composites);, 

2. BENE (employee Benefits); 

3. CONT (contractual services);:- 

4. TRAV (travel expenditures); 

5. .CAPI (capital outlay); and 

6. :.;-SUPL (suplies and materials). 



Di|criminarit\fe-predfction yielded 3 oveiT-estimates and 2 under- 
estimates for a ^totaL^accuracy^f actor of 83>87 ofierce^^^ 
' ' ■ ^ijv. seem, that the criterton 'distributions are m^^^^ in 

^.predicting indi-vldual inclusion, then in determining total inclu- 
Sfior^ across-, ull criterion objective functions. 

NQn-solut-\on results . As was evidenced in the restricted 
residlts, only minimization within a relaxed region' 
produced irtstancel? (3) of non-solution; they^were: 

• ' / V 1.;- CLAS XclVssif led salaries)-; ^ ^ 

2. CAPr\( capital outlay); and 
/ 3. \^,C€MP (tiUd^ktary composites). 
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VALIDATION OF THE R0L8AK MODELING FRAMEWORK 



A total of 31. program ibudgeting (unit) alternatives were eval- 
uated for defunding 'across a total of 10 competing criterion 
references. In lieu of a step-wise procedure as represented in 
•the school closure modeling framework, the model is further 
cons.tr^ained to, choose thos^'^^^irograms for rej[unding suciv, that th^ 
newV.pperating. dj^ is not less th'an 675,000 dollars, 

but not more than 700,000 dollars for the particular ^prpgrams 
under scrutiny. To study the effect of the model's solution 
generation process, the feas:i'bility/ region as defined by the 
constraint matrix and the RHS'-*values is constructed in two 
distinct patterns': a highly restricted region in which very 
stringent controls are ^defined for the modeling procedure; and a 
relatively rel axed re^-ibn in which less striYigent controls are 
modeled. In addition, the ROLBAK forjnulation is executed both* 
for cyclic maximization of the objective functions, and for 
'tyclic minimization af^the objective functions. Thus, a total of 
£ tracking matrice containing 10 potential . .^dlu^^^ sets (each) 

result.'- o . . ^ ■ 

.. *.■■,'■> '' ' .. ■...-jV/v' 

., .;' • ■ ' . ^ ' ■ ' ^ -/.-/^ ' -^y 

This particular m^^ application represents \tHe- "scaht" 

matrix case,' in that a high proportion -(48.7 percent) of cri- 
terion matrix cells contained a * zero 'entry, signif-ying no cost 
"for that particular alternative within a specific object-' 
expenditure category. For the SCHCLO model, the criterion matrix 
was "complete"' — all cells contained a value greater than zerol 

Under the 'restricted' formulation, the 17 resulting solution 
sets signify only 2 distinct solution vectors. In contrast under 
the ^relaxed' formulation, a total of 17_ distinct solution vec- , 
tors result. Under loth restricted and relaxed limitations, 2 
objective functions were unable to declare optimality due to the 
inability to find an initial integer-feasible solution. 
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<Figure 22> and ^Figure 23> display the solution sets 
resulting from optimization within the restricted region 
environments. The selection tally vector is noted, as well as;' 
the impact upon the total budget. based upon the simulated cuts - 
: (i.e., where X=funded). As can be easily seen, the solutions ' 
■fresufeting from optimization vithin the restricted environment 
present only two, distinct alternatives for later discriminant 
'anktyses. . • . " ' ' 

<Figure 24> and <Figure 25> display those solution sets, 
.resulting from th,e' optimizations within a relaxed environment. A 
'rltotal of .17 distinct-'splution s vectors are formed; .-^hd thus 

• the selection tally matrix demonstrates greater variab3''l%y'cthan- 
existent within the restricted orientat1.on;^v. 

■i^;^^^^^^^ functions were^xbnipLit^ modeling* 

:'setti:ng^^^:,f^ requiring a separate disbf-iminant execution for 

• e^ch.bf the distinct, solution vector's .resulting from the'MAM 
;^.^na1j)^sis. As noted in an earlier section .to this paper, cri >- 
^ teribn strength was evaluated utilizing the three composites 
Vect'orsV . V , , ' 

DISCRIMINANT CRITERION INCLUSION VECTOR " 

DISCRIMINANT CRITERION ENTRY VECTOR / 

DISCRIMINANT WEIGHTING SUMMARY VECTOR. 

The first vector .is composed of binary (1,0) "entries 
signifying whether a specific criterion was entered into* the 
discriminant analysis for explaining the variance within the 
solution set. The second vector contains entries of, 1,2,3,... , ' 
such- that the order-of-entry for the discriminant criteria is 
represented. Finally, the third vector contains a factor-weight 
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entry for eaph, of .-.the 'entered' vectors, =*to measure the. relative 
importance of "each of ^th.^ discriminating criterion references. 

The notion of decisioning reliability was evaluat<ed*uti li zing 
two techniques: ' 

CANONICAL .CORRELATION' 4 ' 

• . ■ . • 15 

• RE-CLASSIFICATION ANALYSIS. 

<Figure 26> contains the discriminant results* for solutions 
accountable to maximization within a relaxedv regi on. The first 
ten columns contain the information from the discriminant analy- 
ses for each of the ten simulated solution sets. The ordinal 
numerals represent order-of-entry, whi 1e. the bracketed entries 
[x.xx] contain the factor-weights computed from dividing each of 
the standardized canonical discriminant coefficients by the 
smallest such coefficient for each discriminant analysis. For 
example in the first column signifying the results of discrimin-- 
ating the solution computed from maximizing certif i.cated 
salaries', 5_ criteria were required to. explain available variance 
within the solution, set. * The criterion 'budgetary composites' 
was entered first, and, represents abactor of 2.51 in its impbrt 
ance to the remaining 4 criterion discriminants. The criterion 
'certificated salaries' was entered secondly, and. represents a 
factor of 3.17 in its .relative importance for discriminating the 
solution set being analyzed; and so forth. The selection tally 
vector is similarly analyzed via discriminant functions. 

For understanding the dimension of decisioning reliability, 
computed carlonical correlation coefficients existed as follows, 
for maximi zed-relaxed solutions;. 
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Objective 


Canonical 


Percent Variance 
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Thus It would seem, that a fprmalued objective of ^ 
"maximizing;* the expenditures associated with instructional 
materials in determining which programs to refund during a period 
of scant reso'urses, produced the high.est correlation between the 
criterion matrix .of 10 vectors and .the proposed solution set vec- 
t^r constructed from the, MAM analysis 'execution.' Likewise, the 
maximization of 'budgetary composites'- produced the lowest 
correlation, explaining oflly 53^ percent of independent variance 
within the MAff solution vector . ■ » ."^ 



The second 'phase' of measuring 'decisionirig reliabi lity 
exists in the accura'cy of re-predlcfcing s6luti^l>,Vet mefl^bership • 
ba^^ed updn the ol ass 1fi cation functi oq^. coefficients' generated via 
the discriminant analysis. The bottom portion of F ignite 14 
.portrait these results for each *of the 10 soltition vector's formed 
'by the varying criterion focus of-^the objective func1:ioa. The 
resti5ts of re-cl. assignation for the selection tally vector ^^are 
also disjplayect. . 
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<Figure 27> iKlustrates the similar results from applying 
•discriminant function analyses to the solution vectors formed by 
minimization within a relaxed setting The three vectors for 
denoting criterion strength are easily distinguishable from' the 7^ 
successful (columns) -optimizations. The re-classif ic^tion por- 
tion of measuring decisioning reliability is also shov^/n. 

"The computed canonical correlation coefficients for 
minimized-relaxed solutions: ' - 
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COMP- 



demonstrated, that solution set formulated by minimizing the 
'administrative perceptionV entries in determining a solution, to 
be the best fit with the overall criterion matrix; and the solu- 
tion from minimizing 'instructional materials', the least 'best' 

fit. ^ : * " ' - ^ 

/ ■ . - a 

Regarding the results of optimizing (both maximally and 
minimally) within the restricted environment, <Figure 28> 
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' g '■l!!!!': j "1 g t ^ o« y II c t o r OrJcr-of-^Kntr^, in Ui scr i»»h»ni i n^thc Solution Sot Vector 

El4':^^^-22-.!.'!Si^2ig!j-!^£L^LLJlI^gLi!gj"a" t Crrt""ori"on''£i Vt r y^vJctuT; anJ DiscrimiSaiit 
Weight Lng S»imm3ry Vt'ctorj ' '. 



< VAFUt Of uajEcriVK niNCTiim uuRiia: cyclic-oi^tihizatioh evaluatioiis > 
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Illustrates, the disGriminant function; analysis\fram.ewQrk. . 
Similarly, the canonical coefficients were cbmpute'd- as: 



Solution 
. . .Sector : 



;CanO;nica-l " 
Coeff^icient 



: Of, < 



Percent Variance; 
Explained- : ■ 



Relative 

.Rank ' 

- V ■ ■2;: '■• 




•: . ., .Summary of* Fjttdings 

• ^ • The;,use ofxdAScriminari^^ a Use-RiT post^. 

• V hoc ^'evat U'at ion strategy:: for .^^^ de;di-Vl'bn-makinq 

■ has'-bjp^n studied within' jtwp/'^epar^ settihgli^ the' V 

: schools-; ■and^;thfe^^d^ 

^ ^^^^P^fe'* ^t^- Two' gener'^ljzecyf^s^^^^^^ prHtess'^'wene.. . 

tofn-r^u^ ^^1^ foci: conter^ . 

fiteria^iused to th^; dec^^gro , " 1/ 

^rify the: |eljaBi;ritj^^ the;d^<l^s;raifi 'u^pn'« ; 
criteria uti 





^ that two' reiei^e^ v*^3l:i;t^ 1 ' \ 

Sndy^trjJStworth^^ 



<5 



The author 
njecessary f or^ 
al^^ lity ref ers 

ter i a ' ef f ectedS|^^»!^ff i ons ^^^^^ ^.an^V-p^.; to ' 

what , degree^ d, thi^'^gfi^^ ' vaf^y acrb^k the -r$i^}J^^ 
cycl teal (^gMi^,ations,:XT^ abi 1 ity -i^el^a^ 

study (l)^^^^natitDnsMp. between:: thte - ' o|^^^^ ^ 
(6bjective>f|U||p^ion) and/the resuTts/qf .a :dl3^^ |f^lys^; 
and (2) the i4lationshib.betv/een the . exi,?nt' p^^^^^ regj.orr 
constrai nt ,^ (rel axed restrf^ cted):;.and . the i. results -Of ;a^(|Ti$crj:{rp.fn- 



' f these ends, the multiple linear 

/^I^Jj;'^^^^ functions analysts, is utl- 

*^^^-1i2lsH.to;mea$t^^ the topics of criterion^ strength and decisioninq 
^et^abilitysj . • 



^ "^^^^-^^ analyses illustrate the 

superior efficacy found in^relating multiple correlational stra- 
. Regies to discpvering relationships between solution vectors and 
^^^'$5=^ ^^'^^^ supporting those decisions. 

criterion strength and two measured of deci- * 
'^:;>^- '/'^WWg'^ are illustrated for the reader — all 

' %es horm:ally products of discriminant function(s) 
nation. : • ■ '■ . \ ■ 




It : is ^;a fundamental by-product of this study though all to 
rtant' not to. note,' that the formation of "classification 
'J^^Goefflcie^ the discriminant process provictes an 

excel lent^ way of projecting expected impact from a newly 
gi^l^^.^l'^^-conected-; set t)f data variables. By utilizing the 1 i near . com- 




binationsvof this new data, 'expected correlative.1 decisions can 
be co(pputed which maintain the same variance ^Relationship as the 
decisions; utilized originally in/the initial discriminant 
ana^yses.^ . .. . ''-"^^^-.-^^ ■ . 



' .J the use of discriminant functions in addressing 

the Issues of criterion strength and decision'ing re^l lability, has 
.heeji tllustrated to hold great promise for the decision-maker, • 

^..evaluator; and otherwise problem-solver. Increased 
accountability, visibility and responsibi 1 ity are the maximized 

' ends^'- / ' 
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SUMMARY OF THE ;" MAM" FRAMEWORK AMD "ROLBAK" . ' 

■ 

"'this study has sought to demonstrate the utility of the 
"multiple alternative modeling formu^lation (MAM) in determining 
program units, for continued funding duripg a fiscal, crisis.^ 
Base.sl> upon an acceptance of criterion-referenced modeV for simu- 
lating future, probable decisioning alternatives, the MAM fiscal 
model, ROLBAK , evaluated various forms of data under different 
system goals (constraints). In order to observe the effect upon 
decision-making; that is, which progr'am units to continue, and j 
which to deallocate. LiKe the school closure and curriculum 
activity packaging models preceding it, this fiscal, roll-back 
model will assist program administrators as they seek to continue 
program operation at an optimal levels though in a state of 
reduced funding. 

The Multiple-Alternatives Formul ization 

The multiple alternatives model (MAM) has been deviled for 
the situations' in which mult1pj[e solutions are required. School 
closures require more than one site be selected, to remediate ■ 
existing declining enrollment impacts and wastage "of low per- 
caplta expenditures."' Curriculum activity packaging requires the 
best possible mix of instructional activities to match desired 
outcomes. And, funding crises require, some select number of 
prpgrams be designated for discontinuance. 

The MAM concept models these evaluation complexes through 
the use of systems of linear inequalities and equalities. Each 
inequality (or equality) represents a Sf^^^^if/c" object i.ve pre- 
defined by the' decision-maker; criterioir*referenced and labeled a 
constraint (to final- solution selection). The system of ine- 

.;■ 362 ■ ' , V'- ■ ' ' " ' 
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qualities and equalities relate each constraint objective to each 
of 'the decision a^ltsrnatives being mod'eled (evaluated for ptoen- 
tia'1 inclusion with the final solution 'set) . In add.ition, some 
one or se.veral criterion vectors is (are) selected to act as the. 
overall guide to decisional optimality, as the objective 
function. 

The ROLBAK Multiple Alternatives Model " ' ^ 

The ROLBAK modeling structure studied within this paper, 
presents a MAM-adaptation to. assist decision-makers when program 

areas must be 'cut''(1.e. deallocated) due to reduced funding. 

a . ■ ' . ■ ' . ■ ■ ■ ■ ' " ■ • . ■ 

ROLBAK exists as a sane and rational alternative to the usual 

■percentage-cut across-the-board; and allows the administrator to 

systematicaTly criterion-reference such complex decisions. 

Criterion-referenced constraints have been shown to poten- 
tially include budgets by object classification, surveyed percep- 
tion of affected participants, and. total budgetary compo,s;ite 
control . . Irj, addition, the utility of . varying writer ibrivContV^pl 
(objective fifihction) has been illustrated. 

^ . ■■:~:-;.;v'.^-: ^ ' ■ " ■■ ■■■ ■ 

, Complex Approaches to Complex. Issues 

The authors maintain that issues' involving many potential . .- 
.. solutions are indeed too complex for the human mind to 
comprehend^ Main-effects and. interactive-effects modeling simu- 
:lations' pr'ovide a valid and reliable methodology for evaluating 
the MAM environment. Without such formulations, ^ complex 
. decision-making is little more than l-part "experience" and " 
; 4- parts "blind luck'l''( and often with/less than successful 
results) . — 
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' But the mafn areas of criticism wilM still prevail. First, 
that the need to quantify. the criteria- requires a greater com- 
mittment to criterion-referencing than many decision-makers 
possess; Secondly, that high degrees of time, effort and 
sophistication ^re required of individuals .who psosess little of 
, the abQve. And finally, that the system requires pptimalTy a 



7 ; 



^cpmp}}S:eri and human-based solutions should ' never.^ (?) be base'd 



■ upon cbfrifeuter anal.ys i s . 



* As social scientists. and humans, simultaneously , we 
acknowledge thesemisgivings for what they are; and disagree ami- 
cably (sometimes). > ' ' 



The Future of MAM Design 



The matching of:, micro-computerized hardware and software; to 
desired instructional objectives; /bhe evaluation of item analysis 
techniques yor desi gning' comput.er^assisted: survey techniques; and 
the consolidation of school districts — are a relatively smal.l ^ 
but represGintati ve.samp^ of areas where this author is 
currently develo^vng future MAM applications. Wherever a poten- 
tial for multiple solutiohs exists^ the multiple. alternatives • 
modeT;wiJl ire .there. 'Multiple alternatives model ing •i-S' not the 
wave of the -futurfe — it" is: the available tool of toda^ '- 'po ' 
itM'l*'" ■ ' - ■ ■ ' ■■ , ■ ^ .'. 
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PART IV 



FUTURE EXTENSIONS AND SUMMARY 
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CHAPTER , 10 



THE DI3SEMINATI0N STAGE 



> ' ■ ^[^Extending the Multiple Alternatives Model to the 
-Solution of Other Educatronal Problem Situations 1 
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: EDUCATIONAL MODELIMG AS- THE NEW FRONTIER 

As recent as 'IS/S, f Specul'attroh. to; the- pds^^^^^^ pbtenti aVv; 
of the mul t i pi e al te^riatl v.^s . mdde-1 i n g f r am^wbrk f or pr obi em re>t, 
solution in the fiel»d of educatioh;_^a^^ decidedly; ia^^^ 
cynic. ■ The., idea .of formulating a detai led maihematicaT' model^ 
a.'decisioning problem,..wherein a num'ber. of, .plausibile'alternativS^eS- 
were to b& evaluated across a set of competiti ve^inteT^est^^ 
Awas. to say/the/ least not exactly embraced/- by. the :leaders:hipf- i ri ^ . 
education/'' . ■ :■ ■ -.-/.'yr T ■t/-',,.-. ./ V-^-V.:- •'■ 

^ ■ The opportun Tti es fx)r expl oitati on ,,pf:.'th€-^ model i ng. framework , - 
were mahy-and obviousi and e'diicators vwere'most/^^^ to us:e 

a new technique; f or -solvi ng. a:ge-6id. -rprobl-ems .especially in '.a- ■ 
climate where educators" in' general v/^ held. in as high a. 

professional regard as they mi^ht 1 ike 'or , deserve. . But the If^:^^^. 
problems which were consistent with a ^riultipTe-al ternatives,-. p.hilr' 
osophy^'weife hot., adequately resolvable utiVizing/tbe bid technique^ 

The prime example* was the .manner qn .which elementary schools 
were /evaluated for potential- closure, once -^he policy .deti si on' ;• 
had been^mad.e to close, schools as ; a means of re'sdlvnn^ problem's^ 
associated, with def lining- element aV^y-schooT- /Many ■'of ' 

; the. decisi onal strategies, ranged from the su^knteh^ . 
the .sit^S" based total ly uppa political ly-motlvapd -pri ^ to • 

the 'Super i ntendent charging 'a. corwittee. of lay--perspns to^not only 
develop the criteria for detefmining/potenti a bat ■^}io.^ ■ 

■ to analyze each sivte- and^submit a ifst. of pri pritized scfiool s f or 
sthool board approvaK v . r ■ . ; . 

, ■ ' ' . ' ^ ■ ' ■ . " ' . ' ■.. ■' . • ■ •'. .-r" ■ ■' ■ •■ . - 

■ . ■ ■ ■ . . . . , ■ 

' ^ This is not to. say however /that tlllre. were no ;professionaW . 
and/or ethical attempts to^Tational ly deal 'wi th th^'^'sch^o^^^^^ vCTosure 
issue. But, these very*" spotty attempts were often . tvvarte^iwi^..^^^^ 



efforts of other 'professionals' who would hide their own politi- 
cal motivations behind the vail of protecting a school for such 
reasons as historical preservation, the protection of the 
neighborhood-school concept, or the predi^fion of disasterous 
effect, upon the child's psycho-social'development if it became 
necessary for the student to have to transfer to a new school and 
make new. friends. Unfortunately, many of these efforts over-rode 
the attempts to decrease surplus sjites — schools which have con- 
tinued to drain district mai njjM and instructional 
budgets ever 5.ince; TiS^^II^^^-- ' ^ 




In 1979, one district saw the. high utility for such a complex 
jyypodeling framework and its assistance to educational decisioning, 

*and permitted the initial field studies and validation tests 
^-|^hich were to prove the MAM ajjproach tb .school closures as a very 
'ncrviable'^and reliable strategy (Wholeben, 1980a). Since that time, 
more applications of the multiple alternatives model and their 
utility; for educational decision-making have become accepted by 
the educational leadership. As as been discussed previously in 
this text, the ROLBAK formulation^ (determining fiscal roll-backs 
during educational funding crises) and the MICROPIK formulation 
(evaluating computer-assisted instructional software and micro- 
computer hardware against curricular instructional objectives) 
have demonstrated furth^ applicability for the MAM framework in 
the problems facing the educator today. Other work such as the 
development of the CAP formulation (curriculurn activity packaging 
for instructional development; Whpleben, 1980b)have also illus- 
trated the diverse utility of the multiple alternatives approach. 

. ^ - . , .» 

\ ■ • - 

• . In this chapter, the reader will be introduced to several 
Other MAM applications currently under study and experimentation 
within the educational community. This paj'ticular portion of the 
text will^be necessarily general and non-det ai 1 ed; • and • a more 
jsurvey-orientation will be utilized in displaying various models. 
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For those of you who have survived this far, this will hopefully 
pro\4j< 
book. 



pro\4jcie''^ the needed encouragement to finish the remainder of the 



The next^^section will focus mainly upon the wide-array of 
applications currently be developed. TUen, three sections will 
focu^^.more specifically upon three- of these MAM- formulations. 
FinaUy, the chapter will end with a few, "chosen words on the 
barriers facing MAM. uti lization which may never be resolved. 
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CURRENT DRAWING-BOARD DESIGNS 



'someone very wise and knowledgable, and whom I am sure was 
veryVfamous (although I am at a. loss to remember a n^iTi'e^f;or legal 
citation) — nevertheless once remonstrated: . 



"A professional . is a person who ' having ^ome 
unique and viable skill, carves o,ut a problematiiz 
situation existent in society, and proceeds to 
'demonstrate that this very individual skill is 
, the preferrecl solution technique in all cases." 



m 



■ Having' said the obvious with the likewise obvious intention, let 
us. now proceed to explore several of the multipl6-.altern.ati ves' 
applications currently under exploration and/or development. 



The "Good Times" " . 

i J" ■ ' • 

One can remember past years in education where the concern 
.was not where the next dollar was to come from, or whether or not 
the school could stay in operation for the full term of the year 
— but rather, whether the school would be able to legitimately 
expend all of the fiscal resources made available tont. Many 
educationar administrators (as well as many others) -bec^e very 
lax during these years of plenty. We forgot the vaf ye ^^^d^^^^^ 
work and merit, and the opportunities available through "increased 
productivity and accountability. 




Those days of harmony wilWeappear in the* fu^e; but they 
will not come (or remain) without a restructuring^ the* means, 
by which educators currently conduct their business. 
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In anticipation of the return of the 'good old days' (?), 
several models are currently under design to assure that the 
, prosperity which accompanies good^times, does not likewise lead 
to the decay* brought about by ample^Vesources and technology. 

One such model will assist the educational administrator in 
operational izjng the long-awaited philosophy of inservice training 
for. the classroom teacher ~ that everyone needs renSawal sometime. 
The TCHRHELP formulation (identification of classroom teachers 
for remedial instruction) will. assist this endeavor. 

'i 

Another modeling formulation will aide the building adminis- 
trator and curriculum departmental supervisor to arrive at a valid 
choice for hiring new classroom teafhers in such a way, that not 
only the needs of the curricujar area are satisfied, but that the. 
demands of the school administrator for a balanced staff are 
Ifkewise addressed. The TCHRPOOL model (employment interview 
pooling for teaching candidates) will, meet these parallel n^eds. ^ 

A third use of the MAM framework seeks to provide new direc- * 
tion to the perrenial problems associated with collective 
bargaining and,,hegot1ons. Whether unions see themselves ,as/the . 
protector of employee rights or the facilitator of the edu(^ationa^ 
professionals' development and growth .\. and whether adminstra- 
tors see themselves as management or the facilitator of the • 
satisfaction of student needs and rights ... is not at ilsue in < 
the model (although it should be of issue somewhere). The^ 
NEGOPAK formulation (bargaining package development and negotia-** 
tions process planning) will seek to address the contents of a 
viable negotiable agreement— - what is demanded v. what is needed 
— and what is possible or rational to expect as the end-product. 
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/Of course, education is not under the constraints of what we 
.might call good ^'mest; and currently faces some of the hardest 
decisions educatojys have ever had to make or comply with. Whether 
the United States Office of Education (or more recently, the 
Department 'of Education) was in fact a viable and managable unit, 
js not 31 mo©t issue --^ since it has gone the way of.^he. Department ^ 
of Energy under PnesiSent'Re.agan's New Federalism. Interesting 
• of^, course as a, side-issue only (!.) is the reaction^ of the public 
to regaining the lion's share of their independence — panfcM 

For yearl^ the common cry has been for less federal government 
interference within state educational affairs. . That chance has ' ' 
nov^come bu,t with the opportunity, comes also the mandate to 
conduct business-as-usual on our own volition. Wben -"federal 
interference" stops, so also does "federal financial aide", a 
form of interference no one, really seemed to mind. More seriously 
however, is the prospects for..retention of the many educational 
reforms brought about solely because of federal mandated require- 
ments. Can multiple alternatives modeling meet the test of these 
changing times? ]\ 

Since the implementation of SCHCLO/(the school closure model ) , 
and ROLBAK (the fiscal roll-back model), the obvious , tendencies 
of further design and development has been to meet the demands of 
problematic situations ~ decliniiig enrollments and fiscal bases, 
respectively. Newer strategies (and in some cases, reforinulations ' 
of older strategies) are currently under design for . addressing 
the. issues of maximized output demand with minimized input ayail- 
:ab;i^l?^ty. 

' Jh^fM^EGCLO model (simulating feeder patterns for desegrega- ' 
;t"ion compliance under school closures) is an obvious reworking of*' 




the. SGHCLO formulation, and its interface with: a; nev;er /mod*ef^g 
'Strategy, The FEEDPAT framework (distri-ct tr^ansfer pattferrts^'for' . ' 
^er' school^) . . , . wV - v '* . ' * ' 

« ' » • ■ ' ■ '. ' ' ■ . ■ ' ' • 

ay*s educationa^problems^..;^^^ to th,^ ' l arger 

^^^^^^IHIl^*^^^?^^ Kpvyever • , *.De^ have taken : 

the^^■■^mSi^^^J^; the 'smaller scho^^^^^^^^ 1 especiany 

he. mors rural .ceRt.er"s..; 'Dl^ (attendance ' v 
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v^^^^^ ar-e 'betterv.Pi^S|3dtri^rf for classroom •te^iching,>!^ 

• ^ " . .. aTf^ tfn^at mairay DT :v vet^)q-an.s;,iare in fact:.' dead-Wood'/ : ' 
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invroTve ■the;;evaljjViiQ^^^ p^^p.ryiance, training- and heed •: 
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poli'cii^nalys^^l'or^^^^^ reojfc seeks to eyat;uate ; 
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ifi.f ace of personH^CijCut^^^ ■::V^ V 
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^iep:i,hg^tj,upt(^ K-5; ari^Uhe! resulting 
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'0f / -\.\BO,i:h BlSTMERG 'and'GRADRECON wjll-'be^M d^t^lT 

■ . I ;.;,.'rep[air'\and.;schedu>inq^r> section);; v;,'' ^Xv^^^^ -^ll*'^ -;''; 



;•- There are, of course muTtjple-alternat^ye''decis-^f^^^^^^ ne'eds 
.. .. ■ . WhiQ^i are p^esen.t^vvirhout cohs'i deration of t'be'>iokrit?,of the 
, ■ v.m;, ■^■[^,^s;*.«^,, We ;,w-i 11 ; mention "te samp'»y.,ng,,of th,es.e ''(Ip^lu^^^ and.., 
'', «• il^usti^^te some; of>, thte' eui«^nt M^. f^rmul atj^^^^ and/or 

^ ■ V -^vdeyeTopmsnt stages f of r th^ r^ol^uVT^n^tDf li^r^^-^tirptfSnTati c ori en- 

' • ' ^ • T h Q m or a cti f. \/ . i.K li -f 1 /-I -" « « -i « V. 4^ « i« « j; «V 



- : The |iiecesyi,^^^^^^^^^^^^ u..c 

r*"''^ • <%f 'g;;obleni^...3^?.pctat^^^^ and the 



• •* \^ i^'f'^ ^.■'^l^ "'^s^^f^"ca and,is!eh^yT4^^^ can be a time- 
Gt)|;suming effor*t.^^^ WYjl 'jD:e^^dn'strate^^^^^^^^^^ 

.-'^ 5j^. 'I^IP u^ig, of tlje MaKiX iTOj'dej'i)Tgf;^^ repair ' 

^' * arf'^sche^ulihg), can do aSg/eai^ d^ in sdnfing not only the al- 

. - i' Jocaiitf'Qn , of^. avaiTabl-e inoini'^s^;accora repairs, but 

■ ' ■ ' ^^^^^^^'l^t^''*^^''^^ . '''^ as the prior- 

" I ' sfet (byjiGfe©^^^^^ ' 

: Z ■ ? ^. Where the earlier desiign of lihe'' CAP model (curriculum activi- 
^ V Pack'a^^fg) sought onl.^Q evaju&te multiple curricular o6jec- 
J- fei'V0 and releva^^ns^r.uWional activities for instructional- 
' :y .y .'^SiC^^nq, frhe nllly ^ornujTatett CURRNET model (curriculum net- 
w^^^ the 'i^ull curricular activities of the en- 

■. :■ v:; . t english, social studies, e|c.)- 
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For the'rdistrict administrator who 'has- everything, there is • 
now the' ITEMCHO formulation (itemized survey instrumentatorl which 
designs tornnunity, studefit,- -staff (or whomever) survey instruments 
from a'pool of potential items — according to criterion-related 
needs, demands or circumstances. 




And finally, for the state or federal officials in^^^jlyf^tl on* 
•who constantly complain of their inability to adequately make 
choices with reglrd to full program development (establishment of 
goals, and their delineation into viable action^objecti ves) , or 
with respect to the granting of support monies for special use 
(delineated budgeting development) — there is now the GOALPAK 
(educational systems ' planner and budgeting allocator) modeling 
framework. 

Three of these "drawing boprd" mode]^g formulations will now 
be described in some detail in order to enrich the reader's 
understanding- of the MAM utility within the educational domain. 






In this spctiAn, the use of the muTtiDle alternati vbs model- 
ing formulation will be briefly described :f or • its application t o \ 
the goals f-ound in the consolidati^ori of multiple school districts, 

The'^Context / 



Consider the, potential problems associated with' an att^fempted 
consolidation of the three school districts (A,B^C) depicted below: 
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where, the codes E, J', S, and H represent elementary, junior .high, 
senior high, and high school (9-12), respectively. . 

• * 
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The need .for the' forced-merger 'of or more- school districts 
■n'nto a consolidated district can of ten b^^l inked to financial 
necessity. To maintain a qucil ity instructional program, two 
^'>|liia'll districts g-pn often pool their resourc:es.«i, and ' establ ish tfcie' 
■required' fiscal -support base. Sometimes, the fiscal relationship- 
.jmai^V^^^^^ a distrit:t finds itself, without adequate 

buiidirhg resources (for whatever reason: e.g. condemnation) and /," 
cannot float the necessaryiUnd mcrfiies for construction. Of 
course, •declining enrollments ip' both districts, can be a: hiighly 
motivating factor for merger , .alsa. 

The. issues surrounding, district consol i dation wil 1 themselves 
encompass several diverse areas.. One paramount issue wilt in- 
volve the decision as to whether the existing building sites will 
remain operational (highly unlikely), or whether some will be 
targeted for future closure. Potential closures>tnotwi thstanding, 
attendance areas will require redefinition of their boundaries; 
and thus children who had previously attended' an el ementary. si te 
within their or igi nal district , may now find themselves going to 
school in a cl oser, /though different elementary site within a 
the previously neighbaring di.stricj:'s boundaries. 

From a prograimiatiiC point of .viev/, 'a district merger can often 
require the. re-central izatiori of progrM components within sites 
(centrally) accessible to the majority of children. This could 
mean the relocation of a special education program from one site 
to another -r and across once-existent inter-district bbundaries.,; 
Firially^, geographic as well as prograrmiatic consolidation could 
very likely lead to required R.IFs of" instructional anid support 
personnel. . ' ^ 

* ■ ■ ■ ■ " 

?• ? •: * 

* » - . ' / - . ' . ; ^ ■ ' 




The Alternatives 



. Jhe DISTMERG nd^de.l fhg' formul ation is approached via the land' 
•engineering. technique, called-.'geocoding' . Geocoding simply means 
to divide the geogriphic area 0/ the districts into like-area 
cells (called geocoded. cells, obviously) by overlaying the area 
with a cartesian coordinate graph (shown above by the use of '+'). 

' ' ' ■ . ' 

Each cell (x,y) coordinate represents a segment of an atten- 
dance area or residential partition which must ultimately; be . 
assigned as the principal responsibility of some school. While : 
such criterion, indicators as distance from each cellto each site, 
or number of potential efirollees via each cell is certainly cal- 
culatable — it is likewise unnecesssary. 

Each alternative is structured as the attend'ance cell which 
could reasonably be assigned to a pat^cular school site.' For 
the most part (except in relati^'.ely Targe districts), ttie'main 
consideration will at leas;!: initially be ivifh the elementary sites. 
Thus vvhile (for example) c$!Jl ' [10,13 ] might be assigned to 
either of two- elementary sites, it is unreasonable ./ari.d a. wasts 
of time) to consider the assignment of an attendance cell at one 
end of the map to be potentially .assignable to a site ^at the .other 

The alternatives' structure for a very simple formulation may 
thus appear as follows: " ■ 



j.j ELEMENTARY SITE #1 | j .ELEMENTARY ^ITE #2 ■ j J 
II 1,2 I 1,3 I 1,4] ... I 5,6 || 5,4 { 5,51' 5,6'1 ... j 7,3jl 

11 ' II ' II 

Where attendance cells r 5,4; 5,5; 5,6 ] are potentially assign- 
able to one or the other of the two imposeii elementary sites. 

• ■ ' ■ ■ •■' : ■ ' '. .■ 
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. For more complex formulations, the i^Hity of the FEEDPAT 
(district transfer patterns for .feeder schools) formulation "would 
be required to solve the relationships of junior h1ghs,.etc. to \ 
the elementary sites be evaluated for attendance trahsf errees. 

As will be mentioned below in the subsection concerning the 
formation of criterion constraints, ^he use of composite slack 
variables will be required to maintain^ome basid level of manda- 
tory enrollment for a sfite, such- that failure t© achieve that . . 
.attendance level wilT . signal ,tlje site for potential closure. 

' v' • ' ■ ' 

•■ ■ ^- ' '. ' '.i ' . ' ' ' '■/• ' ■ ■ ' 

. 'The Criterion Constraints ' * \ > 

The DISTMERG formulation- is 'nqt' a simple model to construct, 
although its borrows "heavi ly from its'' predecessor, the SCHCLO 
framework. Initial ly, the physically-oriented ^constraints will 
require quantification (distance of each cell to reasonable sites 
as measured by statute, miles; tinie necessarj^^.i^n tray^rsing the 
statute distance; actual enrollmenj: density of each eel 1 ;. archi- 
tect's recommended capacity for each bui Idlrig; anH so forth). 

..Attendance assigned, to- each building 'must be m©nitor:ed in • 
order to not violate capacity restrictions,, whi le simultaneously 
each and every cell of the districts must be assigned to one and 
only one school. Required travel distance as summed across th6 ■ 
cells will be .miri^mized, while the >as.sign.ed enrollment 'for eich 
^ite will be maximized. The su^n. p.f a counti ng vector, v/here. the 
value '1' is assigned for each/u^^'ciue and indi'vidual' ce]l , -must 
utimately equal the value of t(ie-|iiifiber b total cells assigpSijle 

More subjectivjs criteria may .be;entered Into the model in.'. the 
form of parent's preference for- one sn te over an6t<he»^ as- well as 
the observed instructional -Vrogram of ' a ^^^ijdi rtg. to 

support various desirable, aetivitiesa.; 



■ .' ■ .0-1 . ■ • • : ■ ' • ■ ■ . . 

At the^ same .time, a^constraint ,maxiniizTng^^ physical cover- 
age of the district by, buWdtn^^^^ b^^ required. And the 
need to equalize mi norl^ty-: populate on repre likewis'e : 
be a key . Ingredient of the consolidation-reassignment' decision. . 




4 




GRADRECON ■ THE GRADE-LEVEL RECONFISUR'ATION ■ F^.RMULATION .- ' ■ ■. ; . ; 



A complex- modeling- strategy which rtqufres ..€he .use of slack' 
variables is - the GR'ADRECON formul at ton'. GRADRECON is diVebted-.to 
evaluate. the various grade-level patterns which might be. of -- : ' 
benefit:to the restructuring of ...a school district..; .The' major ' • 
. distinction- between this-rhbdet an^d many of the otlter des'i^ is 
-the flex'ibility surrounding sum-cbmpij ati ons of 'certa'in criterion 
vectors to denote K-5 'v. K-^;- or: §^8 y/.7-8' partitioned -difference 

"^m <■ ■ ■ 



•v'i 

The Alter native^^ 



• As displayed pn the following., page, the modeTln^\stfuctur^ " 
• . ■ ■ ' ■ ■ ■ . ■ ■ ■ ■ - « ■ ■ '■ '4 V . ■■ « " . ■ . • . 

the; multiple alternatives for the. GRADRECON framev/^, is. developed;. 

■ ' ■ -a ■ ■ ■ ' ■ "'■•a-^ ■■ • . 

by creating thirteen (13) storage locations for^ crltfer-ioh data 

representative of eich of the K-12 graale' levels. - tt^e factual - 

decisiomnq varicfbles must thea be constructed as ;c^»tofcisi te si ack§ , 

which wi n under exacuti on cpntai n the" - various-, sufns^^^ - the 

variously partitioned .K-12 grade level s. b * 

-■■>.• ■- ■ ' ■ ' 

' For example, the j and c i'- decision variab^s, will 

contain poent tally .different sums base<^ upon the fact th^t tfie. 

■ K-5 unit will' have grade-values K,!, .>.,5. summed "witf 

storage, location; wh6rpas,the>I<-6 unit will h^ave: .grad^^^^s" 

K,l, . . . ,6 summed within its storage' location^ Likewis.^^^ each 

of the identified si ack sierage composite,.: * *■ 




■ .;The decision to chQbs$;^ai^i<-5. y.-.K-5 -(^^ . - i 

6-8 V. 6-9 in- the 'K-5 setting, as ^opposed to the 7-^ 'v.. 7-9 -in 
"r--- ■ • ■ the K-6^ setting) highlights the extreme complexity ^of the^^ade- 
' : reconfiguration decisi.oning strategy.,- At the, sortie time however, * 
• : . this- complexity also allp^vthe extreme flekibt'Iitiy of/the model ' 

/ . ■= • ■ v ^' ■ : ■ . : • -382 . J--'^'-^ V ' ■ ... i 



ERIC 



to evaluate the 'more normal', combinations of grade levels which 
would likely exist within a single district. ' ' V 

A simple representation of this alternatives' model would 
exist as fol lows:- • •. - 



K 6 6 9 10 K 7 7 9 10 
K 1 2 3 4 5 6 7 8 9 10.11 12 5 8 9 12 12 6 8 9 12 12 

. J'. . ■ : > n 



a =0 



V V ^F~\/ V V ' ' , - 1 



V V V ." , ... -1 " . =0 

. vv V V " -1 %. =0 
,. . . -^^^ 

V V V V _ -1 ' =0 

V V. v _ " -1 =0 

11 1 1 1 =3 

^- ^ _ 11 • . =1 

' ,. ■ 11 =r 

V V V V V v' -1 =0 

V V V - . -1 =0 
, V V V V • , , . . _i ' =0 

V V V V. _ ' -1 =0. 

V . V V -1 =0 

1 1 1 ,. 1 1 =3 

1, 1 =1 

1 1 , =1 

. 1 1 =1- 



where 'v* represents the various criterion values (non-differen- 
tiated) associated .wi%h the required constraints of the decision. 
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Certain field representations ti'ave found, ;that criteria which . 
represent the 'contribul^on' of a particular grade level to the 
overall configuration. best allows the model Us, necessary flexi- 
bility. Because of the structure ^of the model in requiring the 
decision variables to be the slacks, .the ^physical size of the 
formulation itself can become quite /iarge and yet remai.n a 
scant matrix because of its (see abov^) numerous zero submatr ices. 

* I • « 

Although not shown (the author does have some coiTip;^<ssion) , 
additional criterion values could be modeled dtrectlil^S^icthin the 
slack portion of the formulation dempnstrating particular 
contributions of one formulation (e.g. a Ks^S, 5-8, 9-12 as op^ppsed.: 
to a K-5, 5-9, 10-12 formulation biit within the same K-5 settTag) 
over another. . ; 
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f-IAINFIX T THE MAINTENANC.E.-REPA.IR SCHEDULING FORMULATION 



A final representation 'of 'another multipTe aTternati ves model 
within the educational sector remains the use of mathematical 
'modeling structures to decision-maintenance priorities, and tTpe 
resulting schedul ing' of these prioritized repai^r needs.' 

' • • •■^ / 

' ■■■■ ' • " / ' ' 

The Alternatives ;./ • ^ ■ ^ j 



The 'c>)nstructipn of MAINFIX- -fs a unique Contribution of the^ 
MAft framework, since the model is^iinormal ly executed at leastitwo 
separate times. "The. initial 'execution is defined the identifica- 
tion run, wherein the various maintenance alternatives are eval- 
uated, and those fitting within the allocated b'udget are selected. 

f - ■ ^ ' 

If 

The second execution is the scheduling run — that is, with 
the decisioned alternatives as now represente^ti ve of a reduced 
set of alternatives, each alternative being sub-del i-neated into 
three subcategories: "early teK^m, middle term and late term^ as 
concerning the time-of-term for each maintenance unit's repair 
time-scheduling priority. ^ ■ 



The Criterion Constraints 

The MAJ:(NF IX formulation with the exceptance of itsyfe^red 
tw0-pha5.fi execution, is a basic criterion formul ation (pf a full \ 
matrix. Criterion indicators common to maintenance costs, number 
of personnel required, degree of priority, a\iai 1 abi 1 ity of t'he 
necessary repair materials and/or equipment 'are all represented. 

The secondary execution--- where the time for repair is 
d^sion^.as opposed to the^actual * identification gf the intended 
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repair is somewhat more complex, though not as compelling for 
instant insanity as either the DISTMERG. or GRADRECON modal(s). - 
Criterion irieasures of need, demand ^nd subjective priority- are 
input in order to select a scheduling frame of either term-l, 
term-2 or ,term-3 f or^ each and every identified priority. Care 
must be taken of course, to insure that one and only one schedul 
1ng term is decfsioned for each' i dentif i ed alternative. 
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• UNRESOLVABLE ^ARRIER-S - THE l E.QUCAVrQNAL DJLEiyMA • 



A final commentary on /the utility of ^the MAM fi^am'fewbrk* must 
be off erred in-order^ to conclude 'this chapter on a real is^t^^, 
though somewhat pessimistic and downbeat, note. 



A cursory reading of the^Hforegoing material, ^concerning cur-' 
rent developmental arid experimental models, for muVtitfle 4lterna- 
tivas decisi on-making might lead , one to -bel ieve that ;\§alvati;on 
for the educational administrator, is- but, around theHechnol-^gical^ 
corner. Nothing (it is sad for me to say) ccn^ld be^furthe from 
the truth. ' -J " • 




Although it is obvious thai; some ^knowl edge of the modeling . 
concept is prerequisite to successfully-implementing^ the MAM 
orienta1:ion, 'it iS:not as obvious as to the potential barriers 
existent vHthi^ft^ the modeling formulation itself .r- that is, the 
identif ic'atton of alternatives, and the subsequent definition and 
measurement -^of appropriate trTter/on constraints. That most in- 
stitutions of higher education which cred*entia^orr otherwise 
train school administrators today do not off/r courses in the 
applicatloh of quantitative modeling to decisiori-making beyond 
tha^ usual elementary statistjics course required for the masters ' 
degree — signals the unfortunate regarj^many ajdmi^i strati ve J 
trainers have for the use of numbers- in evaluation. 

* - . -' 

fjote the wariness on the part of manj^ educators towards a 
highly structured approach to evaluative Mecisioning (and in a ■ 
sense, accountability), and' the reade^ has. a' very realistic view 
of the usual acceptance of a mathematical modeling framework for 
evaluating and determini'ng final^ decisions. " . . 

An^^if training and experience do not prpvide sufficient, 
obstacles to the use of. modeling for evaluating alternatives, the 
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reader, h^s only to recall /\on/on# hand of jfive frnfgers) .the 




n/ I 



'ear, wl 



vhere decisions were made 



number of times over the pi 

based upon., solely pol i tical.^Bpnsi deratio]>s.; in lieu of "the actual 

enforcement of the data cite^ as the basisjof the decisions so , 
administered, ^ - ^ ' 

,Now i^r the upbeat and positive side,,,, ' ^ ' 

Current liti^gations and court rulings point to a renewed 
interest on the^part of communities for informed .decision-making. 
More and more,^ teachers are questioning/ the actual , criterion 
base upon Which decisions are made and furthermore are examit^- 
ing the decisions made against the criterion indicators held as 



■indicative of those decmtonsV For their part, the emerging 
educational administrator is slowly recognizing tlie very influ- 
ential ro}e of computers as a medium for new forms of evaluation 
and decision-making. ^In' effect, ^the future 0/ education is not 
tomorrow's innovation, but ra.ther the recognized use of • that 
innovation as a means of making education./nore accountcible^ and 
more available to ^external val idation by, tts critics. ^. 
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IHAPTER 11 



THE CONSOLIDATION, STAGE 



C Related Topits and Perspective 3 



V 




COMPUTERIZATION AND AVAILABLE SOFTWARE- " 

^ ■ • 



A common question posed to the author^* from the field as to 
■the implementation of the MAM framework, revolves airound the ob- 
vious requirement for. computer hardware and packaged software in 
ordeV to utilize the mathematical formulation for evaluation and 
^ifJeGi'sioning. ^ - ■ . 

■ \ ■ ' ■ " ' 

.DCrring the .past five years, various modeHng software packages 
have Ipeen .utiliz^ed to differing degrees of satisfaction and ease. 
Invariably, a .'particular soHware package will be har.dware-bound, 
and thus what might have existed on a UNIVAC 1110 will (may) not 
.be available or^ a CDC CYBER. 170-75Q or IBM 370. Although these 
machines are consideVed macro-mainframes, several equally good 
packages are avai'Table on what have come to be known as super- 
miniframes or mini-computers (e.g. VAX 11/780 or HP3000). 

In. this section, we wilV be concerned primarily with the 
software packages the authbr has utilized, although mention. :wiH 
obviously be made as to the type of machine housing the particular 
package. > - ^ ' . 



Macro-Computerized^ Software ^ ^ 

■ ' ■' * . ■ : "... ■ 
The Wadison Academic Computing Center (MACC) at the University 

of Wiscensin at Madi sbi^iKti 1 izes a UNIVAC 1110 macrO' mainframe; 

and supports a wi.de-ar/a)^f ^mathematical prograniming software" 

(of which integer programming routines are but a few). for use as 

mathematical modeliiig routines for the School of Engineering, 

School of Business, and the Department of Computer Science.. 

. One such software package, known as IPMIXD, performs a stan- 
dard solution generation complex by way of a FORTR-AN-based sub- ^ 
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routine. The MAM'problem is modeled vvithin -a FORTRAN-written 

^ program, and the IPMIXD package callei^ a standard subrq^utine 

reference. The output ranges from a few pages to boxes ... and 

the analyst must be extremely careful when assigning initial 

^^a^^eters. IPMIXD is fairly 'powerful, and •provides readable 

output for the novice. • 

» * * 

The University of Washington, (in Slattle) utilizes a rela- 
tively new CDC (Control Data Corpc3ration) CYBER 170-750 number- 
cruncher within its Academic Computer Center ' (ACC) . The 
predominate integer-prograimii ng package is the standard MPOS 
(Multi Purpose Optimization System) package distributed through 
the Vogelback Computing Center at Northwestern University in 
Evanstdn, Illinois. ^ ' 

Unlike'IPMIXD, the MPOS integer' routine (called 8BMIP) is a 
key-word oriented selection of commands — not requiring the 
modeler to first prepare a standard FORTRAN (or ather compiler- 
based .language) program. -'Such a program has obvious/ease'for the 
non-programmer. MPOS provides for e)^ernal datafile access, and 
therefore can be considered extremely flexible, as well as power- 
ful. ^ 

While both MPOS and IPMIXD are both batch-processing oriented 
packages, another software package available on the University of 
Washington macro-system is the interactive package, EZLP. This 
mathematical programming routine was developed at the Georgia 
Institute o^ Technology and also allows i'ndirect datafile access' 
from, an extern]al source. Coupled with the ability to be -accessed 
interactively. from a terminal , and executed on-line — EZLP pro^ 
yides rmmediate feedback and solution formulations for relatively 
medium-sized problems. 

i 

. , ■ 391 . 
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Min1-ComDU*ter1zed Software' 



Developed by Or. Linus Schrage of the University of Chicage, 
the interactive package LINdV (Linear D^iscrete Optimizer) is 
available on such mini-computers as the Digital Equipment Corpor- 
ation's VAX fli>780 and PDP machines. While somewhat easier to 
^use is opposed' to the EZLP inieractive package,* LINDO does have 
,a. major shortcoming— external datafile elements are non-acces- : 
sibl^e to the pir^ogrammi ng routine. Therefore, aH data must be 
• enj/ered in the form of the algebraic inequality, for each and 
every criterion constraint. However, LINDO.is by far, the 
easiest for. the novice-student to learn, and implement. Output 
is good; and wUl be' as annotated for the user as Initially input 



Micro-Comouf erized Software 

— 7 — \ : ■ ■ • ^ • 

To the current knowledge of this author, there exists jio 
integer programming MAM routines for use in such micro-computers 
as APPLE- or TRS-30. However, plans are currently underway by 
the author to develop a moderate-sized routine written .in BASIC 
for the TRS-80 TRSDOS BASIC and APPLESOFT BASIC compilers. 
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PERSPECTIVES AND IMPLICATIONS 



One of the hardest parts of writing a fairly lengthy -manu- 
script is developing a reasonably meaningful las^ chapter. 
Obviously, the more important points coincerning context, design, 
implementation ^nd interpretation have fceen dealt with in' pre- 
vious chapters. At the same time, good authors should somehow 
bade their readership farewell ...^so ... 

This text has been postulated on the^belief, that the role of 
the educational leader in the years to come will require greater ' 
involvement wi-thin problem analysis and resolution from a more 
quantitative perspective. Although educators and other social ^ 
scientists have for decades espoused, that not everything is 
measureable ~ there are nonetheless ample opportunities ' in -which . . 
to develop quantitative assessment strategies for evaluation and • 
decisioning within the educational environment. The mathematical ' 
model ing methodology developed within thi^text is certainly a 
firm example of the power which the 'manager could display in 
dealing with the complex issues of today. ■ * 

While at the University of Washington, I have managed to de- • 
velop a -doctoral . semi nar in which practicing administrators have 
been instructed in the use of the multiple alternatives modeling 
.methodology — a practice I will be able to continue with the \y 
University of Texas ^ El Paso. A copy of the course perspectus 
has been included for the reader's perusal within the appendix 
of this text. 

Yet with the:,eagerness and enthusiasm displayed by the students 
within this seminar, there has als^o been- the willing disregard for 
the model •>g^om some school districts in the area who could have 
availed themselves ^f such a simulative design 'for determining 
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' sgIiooI clos^ures, oT ear-rrrarking prolan] terminations: under fiscal 
recession-. ,.And^eiien^ain, other school di^^tricts have embraced 
the design .with at the least, a need tQ' understand whether^ or * } 
not their criterion indicators will in f act . di^rimi nate between 
poterttiial sol uti oYi , alterhatives: , . 

As -educators' be^e-' more Bccovr>|,able,r.and as school boards 
and community group^idemand Tnore' vi si di 1 i ty and responsibi l ity- 
from 'their ^^ppointe.d. educational 'leaders — the role pf. models 
like MAA wi IT take on new importance '.and Tike. , It is ridiculous, 
to .assume,- . thaf the'^omputer'or >£s associated, software "will ,in 
■effect, make^ the decision modeled^. But,, it is li4<ewise ignorant 
to assume^ 'that .the yse-. of a' quantitative analysi s- framework- is 
worthless since it has hot been widely act^ptaff or utilized. 



Like computer literacy and 'high • school proficiency examina-^ 

^ - ■ . ^ ^ ■'^ 

tions, nlathematical programming and mu.l.tiple alternatives fnodel- 

ing will Gome into its own— not because it was desired or even 

i}ked -- but because it was finally recognized' as unavo^idably 

.necessary. . ' \. '} • 

. ■'. ' ■ ' '>i ■ ■ ■• ' . 
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APPENDIX A: COURSE OUTLINE FOR MAA INSTRUCTION 

<««««««««««««<*»»»»»»»»»»»»» 

SPECIAL PROBLEMS IN EDUCATIONAL ADMINISTRATION .. 
" Evaluation Model inq for Educational Decision-Making ". 
<««««««<<^«««««*»»»»»»»»»»»»» 

This course wilJ examine the role of mathematical modeling 
techniques, and their importance within the realm of educational 
evaluation and decision-making. Class presentations will be 
sequential; and will deal with such issues as areas of technique 
application, general design constructs of the "Multiple 
Alternatives Model" as portraying the execution of "Multiple 
Alternatives Analysis", specific building steps in formulating an 
individual model, computer-oriented techniques useful during the 
design and analysis stages, and required pre- and. post-analysi s 
statistical procedures for assessing content validity and process 
reliability. 

Although course pre-requi si tes are not currently permitted 
for selected entry into this seminar, the student is advised of 
the following assumptions on the part of the professor: that, 
each seminar member have successfully accomplished at least two 
courses in statistics, have at least a rudimentary understanding 
of the formation and uti 1 ization of simultaneous linear equations 
and inequalities from linear algebra, and have had some' prior 
experience with writing and executing SPSS statistical programs 
on the CYBER 170-750 (main cfampus computing mainframe). 
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Because of the sequential nature of the material to be pre- 
sented within the class sessions, attendance at each session is 
mandatory , and without exception except in cases of emergency. 
This rather unusual requirement is demanded due to the complexity 
and technical difficulty' associated withi the course content. It 
has been'found over the past three years, that students missing 
any amount *bf (or ]}ortion thereof) the sessions have been unable 
to complete course requirements. EdAdm 5.37 is an advanced 
seminar; and since the majority of the attending/ students -.are 
pursuing doctoral study, such attendance perogative should be 
assumed a priori . 

N ■ 

Since this seminar is .truly ' field-applications ' oriented, 
individual readings will be confined to those contained within 
the course texts, and those additional library references which 
individual students will have to survey based upon the topic and 
direction of their particular course project. Individual work on 
the defined field project is required; and therefore, no group 
projects will be allowed^ No examinations or tests are planned 
at the coi^clusion of this "seminar. Full basis for the final 
course grade depends upon the results of the individuaT field ^ 
application, and the appropriateness of the final project report . 
Exceptions to *or substitutions for the project and report will 
not be allowed. 

The final report associated with the. field application is 
expected to be a fully-edited, clean manuscript of approximately 
20-25 pages, typed, 1^-spaced, and following the rules of some 
acceptable thesis preparation guide (NOTE: the APA PUBLICATION 
iMANUAL is strongly suggested as a useful and relatively inexpen- 
sive investment, and is available at the campus bookstore). All 
tables a/o figures must' be fully readable, labelled, and set-off 
on separate pages from the general text of the manuscript. 
Manuscripts which would not be considered acceptable for publica- 

\ 
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tion as a journal article, will not accepted as fulfilling the 
course completion requirements. A general topical format for 
your consideration, but only as 'an example, is^ 

ABSTRACT (100 words or less) 
I- INTRODUCTION TO THE PROBLEM . ' 

DISCUSSION OF THE TECHNIQUE 
MODEL DESIGN AMD DEVELOPMENT 
DATA MEASUREMENT AND FORMULATION ' 
" ' " EXECUTION AND RESULTS 

VALIDITY AND RELIABILITY TESTS - . 
DISCUSSION AND SUMMARY 
IMPLICATIONS ' 

Modifications to the above out^|ine are acceptable; but the stu- 
dent is cautsoned to remember, that some of the major criteria to 
be utilized in judging the final report will be those of 
completeness , thoroughness and understanding . 

The subject or focus of the actual field investigation is at 
the discretioni of the student, with approval from the professor. 
The fiejd problem to be modeled must however be of such a nature,' 
that it demonstrates the qualities of a true ^multiple 
alternatives' situation, and that the application of a quan- 
titative evaluatiory- and decision-making model to the problem's 
remediation will result in a useful documentation of content and"' 
process information for the educational field practitioner.. Due 
to the time restrictions placed upon an academic quarter seminar, 
^there will exist some negotiable room between 'what is defined* 
as the bases for a full-implementation of the specific deci- 
sioning (sic) model, and 'what is suff^icient* for empirical deno- 
tation of the validity and reliability issues involved in the 
resolution of the defined problem. 

. 399 



400 



<EdAdm-537> exists as one of the more intense seminars in the 
530-s8r1es, and; demands' an extreme^ almost dogmatic attenti veness 
and partlcipati veness on the part of the student. This par- 
ticular seminar has 'also- been found to be a-most useful sequence ' 
in the .development of skills for evalLfating and resolving many of 
the more c'omplex problems facing the emerging educational admi- ' 
nistrator today. Computer systems are but tools; and quan- 
titative analysis techniques f or -resolVi ng complex issues are but 
skills; you are the catalyst which can make' them work for the 
optimization of educational performance and progress.. 



[Section 01] INTRODUCTION TO MODELING & APPLICATIONS 

: * 1, The Multiple Alternatives Setting 

2. Policy V. Action Alternatives iModeling 

' 3. Main v. Interacti»ve Effects Modeling 

4. ^ngle v. Multiple Entry Solutions 

V, 5. Linear Inequalities and System Impact 

6. Maximization/Minimization for Optimality 

[Section, 02] TECHNICAL COMPONENTS OF THE MAM FRAMEWORK 

' r. Measurement Scaling and Data Generation 

2. Criterion Definition and Referencing 

3. Development of Decisional Alternatives' 

4. i^riterion Comparison Vectors 

5. RHS-Limits and the Conditional Vector 

6. The Constraint Matrix 
7^ The Objective Function 

^ 

[Section 03] CONSTRAINT MATRICE AND CONDITIONAL VECTORS 
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^ 1. The General System Impact Constraint ^ . 

2. The Specific Alternative Impact Constraint 

3^. Modeling the Four Measurement Scales 

4. Search, for Initial Integer Feasil)i I >ty 

^. Relaxed V. Restr-icted RHS-Inf luence 7 

6. Modeling Maximization v. Minimization, Bounds 

[Section 04] X OBJECTIVE FUNCIglONS AND CYCLIC OPTIfCzATION • 

» ■ ■ 

1. Utility of a Cyc+4c-. Optimization Procedure 

2. Development of the 'Tracking Matrix' 

3. ^tru.cture of the System Database 

4. Post-^Hot Assessment of Results 'Per Cycle 

[Section 05] DEFINITION AND DEVELOPMENT OF ALTERNATIVES 

1. Mutual-Exclus.iveness for Independence 

2. Col lecti ve-Exhausti veness for Totail Control 

3. Delineation Paradigm for Alternatives 

4. Single-Entry and Multiple-Entry Definition 
' ' 5. The Hierarchical, Multiple-Entry .Situation 

6. Tautological Comparison and Control . 

[Section 063 DESIGN AND CONSTRUCTION OF THE DATABASE 

. ■ 1. Review of NOS Command Procedures 

' . 2. Introduction to the UEDIT System Editor 

'3. Computing Initial Statistics Using MINITAB 

4. Transposing Col liinn- Vectors to Row-Vectors 

5. Building MAM-Databases Utilizing MINITAB 

5. Modifying the Database with the UEDIT Editor 

[Section 07^ DESIGN AND CONSTRUCTION OF ANALYSIS ^PROGRAMS 
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. * 

1. IntroductibrhNto MPOS Integer Programmi'ng 

2. UEDITing the MPOS Executable Package 

3. SUBMITing, QGETing and ROl/tEing Results . 

~) . ' ' 4. Normal Techniques for Initial Feasibility - 

5. Comparing Restricted v. .Relaxed Solutions 
5.^Comnon Mistakes with the'MPOS Package 
7. Verifying MPOS Analyses and. Findings 

[Section O83 PRE-ANALYSIS STATISTICS FOR MODELING CONTROL • 

\ 1. CORRELATION for Criterion Independence 

' 2". ONEWAY ANOVA for Undesirable Criterion Bias 

^. 3. FACTOR ANALYSIS f3r Criterion Efficiency " 

[Section 09],. POST-HOC STATISTICS Fi5r VALIDITY & RELIABILITY 

. 1/ ANOVA and the Optimality Weighting Matrix 

2. DISCRIMINANT FUNCTIONS and Predictability . 

3. UEDITing the Database for SPSS Analyses 

[Sect ion /10a FORMULATION FOR STEPWISE SOLUTION STRATEGIES 

1. Dichotomizing the Selection Tally Vector 

2. Restructuring the RHS-values 

3. Restructuring the Database via UEDIT Editor 

4. Approaching the Final Solution Set as Limit 

[Section 113 INTERACTIVE MULTIPLf ALTERNATIVES MODELING 

^ *^ ': \ • ' " 

1. The EZLP Package 

2. The'Ll'NDO Package 
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APPENDIX 8: COURSE SYLLABUS' FOR MAA INSTRUCTION 



< Genera>1 Topio <Spec1fic Reference> <Instructiona'l Objectives> 

— ^— . , I — — 

. MULTIPLE ALTERNATIVES CONTEXT 



school closures 

curriculum activity packaging 

pi^- employment teacher inter vi.ew 



am ng . 



4^ y 

8 
9 

10 



microcomputer hardware/software matching 
budgetary rol Tracks 
sphooT district consol idationis 
? attendance boundary red^i^stricting 

elementary school grade-leveT reconf iguation 
feeder-pattern control 

survey questionnaire item-ranalysis and selection 



GENERATING MULTIPLE ALTERNATIVES 

1: delineation of objectives' framework 
'brain-storming' framev/ork 
'random-components' 1i,]^ework 



DEFINING CRITERIA ra^AM ANALYSIS . . 

1: generic* client ati on 

(eff ecti veness, efficiency, sat isf action, expenditure) 
2: species'^'orientatio.n 

( intent, met hod, value, means) 
,3-: measure orientation 

V (physical, trafisformational, definitional, synthetic) 
^" ' \ * * ' 

DEVELOPING CRITERIA FOR DATA COLLECTION 
1: documentation ^reviews 
2: survey col lection 
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SCALING CRITERIA FOR DATA MEASUREMENT 
1: 'common scalingl-techniques 
2: counting^techniqu^s 
^ 3: weighting techniques 
4: probabi listic-measuremant techniques 

FORMULATING LINEAR- INEQUALITIES 
1: vector summations 
2: slack storage units 
3: tautological decisioning vectors 

COMPUTING THE CONDITIONAL VECTOR^ 
1: rav/-score summation technique 
2:"" t-normal generalized affecr technique 
3: differential impact technique ^ ... 

(mean w/ standard deviation) 
4; s^Decial case of the 'scant' conflict matri 

PREPARING THE OBJECTIVE FUNCTION' 

CONSTRUCTING THE DATABASE 

1: "access to the CYBER 170/750 

2: . use of the MINITAB package . ■'. 

3: T-Normal transformations 

4: matrix, transposition procedures 

5: conflict matrix generation^ 

STATISTICAL ROUTINES TO PRE-ANALYZE CRITERIA ' 
1: Product-Moment Correlation . 
2: Non-parametric Correlation 
3: Oneway Analysis of Variance 
• 4: Factor Analysis . ' 

MPOS(BBMIP) ROUTINE TO )\NALYZE MAM PROBLEM 
1: access to the CYBER 170/750 




2: Northwestern University MPOS Routines 

3: use of th^ BBMIP integer programming ^routine 



STATISTICAL ROUTINES TO POST-ANALYZE SOLUTION(S) 



Oneway Analysis of Variance 
T)iscriminant Function Analysis 
^Multivariate Regression 



CYCLIC OPTIMIZATION AND THE TRACKING MATRIX 

• 1: dichotomy-scaled selection techniques 
. t 2: observation^scaled selection techniques 

STEPWISE SOLUTION(S) 1|^D^ RJS-UPDATING • 

MODEL RELIABILITY AND SOLUTION VALIDITY 



r 
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APPENDJX C: . PROPOSAL OUTLINE FOR MAA FIELD STUDY BROJECT 



CONSTRUCTING THE EDADM 508 PROJECT PAPER 



<A Guide to Content and Sequencer 



Item 



Delineation of Contents 



ABSTRACT OF PROPOSAL 



1. 350 words or less, in 3 paragraphs , 

2. 1st paragraph = statement of mission 

3. 2nd paragraph = statement of method 
;4., 3rd Rig^agraph statement of results 



TITLE PAGE 



1. title of project, with short 1-sentence 
description of project 

2. authorship credit, with credential 

3. statement of- funding credit 

4. statement of disclaimer 



TABLE OF CONTENTS i-'XA. all major topics, and main subtopics, 

i< by page reference ' . 

/ . ; 2. appendice reference by ^ page number - 

3. list of attachments ' ' 

IIST-OF FIGURES/TABLES 1. full title of figure and/ or table 

^ with page reference 

INTRODUCTION TO PROPOSAL 1. general introduction to the proposed 

program and/ or project 

ASSESSNlENT OF NEED(S) 1. design of the assessment, with statement 

- of 'significant others' participation' ^ 
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2. rationale for data collected, analysis' 
interpretation(s), and decisions drawn 

3. specific reference to the constructive 
' i nt err ogativ^ adverbs' of WHO, WHERE, 
WHEN, HOW,/^HY, and WHAT a., 

4. discussion of miRtiple alternative 
strategies toward problem resoTution, 

and the (suggested) strategy to reconcile 
direction 



STATEMENT OF PROBLEM 



1. 1-page discussion of the perdeived 
a problem; more of a narrative summary 
of the "assessment of need(s)" section 



'STATEMENT OF MISSION 



1. 1-page statement of the intended direction 
of the remediation tactics to be 
developed within the remainder of the 
proposal 



DELINEATION / OBJetTIVES 1. utilize the format of: " - 

phase. goal .objective. activity. task 
. 2. structure this section in outline form:* 

01 Statement of Phase ' 
01.01 Statement of Goal 
01.01.01 Statement of Objective 
^ 01.01.01.01 Statement of Activity 

01.01.01.01.01 Statement of Task 



NARRATIVE'DESCRIPTION 



1. narrative explanation of project 

2. review of si^pporting rationale for 
' decisions made during design and 

development 

3. explanation of time and other resource 
allocations . 
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4. explanation of responsij3l^I ity -lissilgnments 

5. summary via GANNT Chart 



FORMATIVE EVALUATION 



1. e)5planation of framework suggested to 
conduct ^program improvement' evaluation 

2. definition of criteria to be measured 

3. ; discussion of measurement schem^es'for 

data colleoxion ^ ^ • 

4. denotatifin of major 'milestones' where 
significant departures from expected 
schedule will require a restructuring 
of the propose(;l program / 



SUMMATIVE, EVALUATION 



1. explanation of framev/ork suggested to 
conduct 'performance output' evaluation 

2. definition of criteria to be measured * 

3. discussion of measurement schemes for' 
data collection 

4. suggested summary reporting procedures " 
for the assessment results to each of: 

a. effectiveness (intent) 

b. efficiency (method) ' 
.c. satisfaction ■ (value) « 
d". expenditure (means) 



REPORTING OF RESULTS 



APPENDICE 



1. exp^lanatfon of^ the structure for the 
final document to_ be submitted at the 
conclusion of the program's implementatib;i 

2. introduction of 'sample* reporting* 
figures, tables or graphs 

1. copy of goals/objectives flowchart 

2. copy of activities/tasks network 

3. 'summary of activities' pages from the 
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ATTACHMENTS 



CPM, computer analysis " 
4. dayfile page from the Computer run 

1. any supporting doeumentation, 'figures^ 
or .'other* material's, -which are germane • 
to presentation of th^e proposal but not' 
nec^^s^ari Fy a formal ji^t .of the program 
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